{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# REINFORCE Policy Optimization with PyTorch\n",
    "\n",
    "In this notebook, we will look at policy optimization using PyTorch. Let us quickly go through the derivation of what is policy Optimization and then we will apply this to CartPole environment - just like what we did in Chapter 6. \n",
    "\n",
    "\n",
    "### Derivation of Policy Gradient\n",
    "In Policy Optimization, we will have a neural network which takes in state `s` as input and produces the `logits` for action probabilities. \n",
    "\n",
    "The policy is parameterized by $\\theta$\n",
    "$$\\pi_\\theta(a|s)$$\n",
    "\n",
    "The agent follows the policy and generates the trajectory $\\large \\tau$ \n",
    "\n",
    "$$ s_1 \\rightarrow a_1 \\rightarrow s_2 \\rightarrow a_2 \\rightarrow .... \\rightarrow s_{T-1} \\rightarrow a_{T-1} \\rightarrow s_T \\rightarrow a_T$$ \n",
    "\n",
    "here $s_T$ is not necessarily the terminal state but some time horizon T upto which we are looking at the trajectory. \n",
    "\n",
    "The probability of trajectory $\\large \\tau$ depends on the transition probabilities $p(s_t+1 | s_t, a_t)$ and the policy $\\pi_\\theta(a_t|s_t)$. It is given by the expression:\n",
    "\n",
    "$$p_\\theta(\\tau) = p_\\theta(s_1, a_1, s_2, a_2, ..., s_T, a_T) = p(s_1)\\prod_{t=1}^{T}\\pi_\\theta(a_t|s_t)p(s_{t+1}|s_t,a_t)$$\n",
    "\n",
    "The expected return from following the policy $\\pi$ is given by:\n",
    "\n",
    "$$J(\\theta) = {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\sum_{t} \\gamma^t r(s_t, a_t) \\right]$$\n",
    "\n",
    "We want to find the $\\theta$ which maximizes the expected reward/return $J(\\theta)$. In other words, the optimal $\\theta=\\theta^*$ is given by expression\n",
    "\n",
    "$$\\theta^* = arg \\underset{\\theta}{max}{\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\sum_{t} \\gamma^t r(s_t, a_t) \\right] $$\n",
    "\n",
    "Moving on, let us try to find the optimal $\\theta$. To keep the notations easier to understand, we will replace $\\sum_{t} \\gamma^t r(s_t, a_t)$ as $r(\\tau)$:\n",
    "\n",
    "$$J(\\theta) = {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ r(\\tau) \\right] = \\int p_\\theta(\\tau)r(\\tau) d\\tau$$\n",
    "\n",
    "We take the gradient/derivative of above expression with respect to $\\theta$:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\nabla_{\\theta} \\int p_\\theta(\\tau)r(\\tau) d\\tau $$\n",
    "\n",
    "By linearity we can move the gradient inside the integral:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\int \\nabla_{\\theta} p_\\theta(\\tau)r(\\tau) d\\tau $$\n",
    "\n",
    "Using log derivative trick, we know that $\\nabla_x f(x) = f(x) \\nabla_x \\log{f(x)}$. Using this we can write above expression as:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\int p_\\theta(\\tau) \\left[ \\nabla_{\\theta}\\log{p_\\theta(\\tau)} r(\\tau) \\right] d\\tau $$\n",
    "\n",
    "We can now write the integral back as expectation, which gives us the expression:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\nabla_{\\theta}\\log{p_\\theta(\\tau)} r(\\tau) \\right] $$\n",
    "\n",
    "Let us now expand the term $\\nabla_{\\theta}\\log{p_\\theta(\\tau)}$ by writing out the full expression of $p_\\theta(\\tau)$. \n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\nabla_{\\theta} \\log{ \\left[ p(s_1) \\prod_{t=1}^{T}\\pi_\\theta(a_t|s_t)p(s_{t+1}|s_t,a_t)\\right]}$$\n",
    "\n",
    "We know that log of product of terms can be written as sum of log of terms i.e. \n",
    "\n",
    "$$\\log{\\prod_i f_i(x)} = \\sum_i log{f_i(x)}$$ \n",
    "\n",
    "Using the above substitution, we get:\n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\nabla_{\\theta} \\left[ log{p(s_1)} +  \\sum_{t=1}^{T} \\left\\{ \\log{ \\pi_\\theta(a_t|s_t)} + \\log{p(s_{t+1}|s_t,a_t)} \\right\\} \\right]$$\n",
    "\n",
    "The only term dependent on $\\theta$ is $\\pi_\\theta(a_t|s_t)$. The other two terms $log{p(s_1)}$ and $\\log{p(s_{t+1}|s_t,a_t)}$ do not depend on $\\theta$. Accordingly, we can simplify the above expression as:\n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t|s_t)} $$\n",
    "\n",
    "\n",
    "Substituting the above term into the expression for $\\nabla_{\\theta} J(\\theta)$, as well as expanding $r(\\tau)$ we get:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\left( \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t|s_t)} \\right) \\left( \\sum_{t=1}^{T} \\gamma^t r(s_t, a_t) \\right) \\right] $$\n",
    "\n",
    "We can now replace the outer expectation with an estimate over multiple trajectories to get the following expression for the gradient of policy objective:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\frac{1}{N} \\sum_{i=1}^{N} \\left[ \\left( \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\right) \\left( \\sum_{t=1}^{T} \\gamma^t r(s_t^i, a_t^i) \\right) \\right] $$\n",
    "\n",
    "where i denotes the $i^{th}$ trajectory. \n",
    "\n",
    "To improve the policy, we now take a +ve step in $\\theta$ in the direction of $\\nabla_{\\theta} J(\\theta)$:\n",
    "\n",
    "$$\\theta = \\theta + \\alpha \\nabla_{\\theta} J(\\theta)$$\n",
    "\n",
    "To summarize, we design a model which takes state $s$ as input and produces the policy distribution $\\pi_\\theta(a|s)$ as the output of the model. We use a policy to generate returns and then change the model parameter $\\theta$ using the expression: $\\theta = \\theta + \\alpha \\nabla_{\\theta} J(\\theta)$\n",
    "\n",
    "\n",
    "### Rewards to Go Trick\n",
    "\n",
    "\n",
    "we drop the reward terms that came before time t as at time t, the action we take can only impact the reward which comes at time t and later. This leads to changing the 2nd inner sum going from t’=t to T instead of earlier sum over t’ going from t’=1 to T. i.e. the start index is now t’=t and not t=1. The revised expression is given below:\n",
    "\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\frac{1}{N} \\sum_{i=1}^{N} \\left[  \\sum_{t=1}^{T}  \\left( \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\sum_{t'=t}^{T} \\gamma^{t'-t} r(s_{t'}^i, a_{t'}^i) \\right) \\right] $$\n",
    "\n",
    "\n",
    "### Implementing Loss and Gradient Step in PyTorch/TensorFlow\n",
    "\n",
    "We will implement a pseudo loss function, whose derivative will give us $\\nabla_{\\theta} J(\\theta)$. Also as PyTorch/TensorFlow carryout a gradient Step, we will convert maximization to minimization by changing the sign of this objective function\n",
    "\n",
    "$$L_{CrossEntropy}(\\theta) = - J(\\theta) = - \\frac{1}{N} \\sum_{i=1}^{N} \\left[ \\sum_{t=1}^{T} \\left(  \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\sum_{t'=t}^{T} \\gamma^{t'-t} r(s_{t'}^i, a_{t'}^i) \\right) \\right] $$\n",
    "\n",
    "To summarize, we will pass the state `s` through the network to get $\\log{ \\pi_\\theta(a_t^i|s_t^i)}$. We will calculate the cross_entropy loss for the actions actually seen in the trajectory. We will then calculate the weighted mean of these individual loss terms in the trajectory with weights being the rewards-to-go $\\sum_{t'=t}^{T} \\gamma^{t'-t} r(s_{t'}^i, a_{t'}^i)$\n",
    "\n",
    "This will be followed by a gradient step in -ve direction of weighted NLL (negative log loss) i.e. in positive direction of the gradient of $J(\\theta)= - L_{CrossEntropy}(\\theta)$ \n",
    "\n",
    "We also add a regularization term known as Entropy. Entropy of a distribution is defined as:\n",
    "\n",
    "$$H(X) = \\sum_x -p(x).log(p(x))$$\n",
    "\n",
    "To keep enough exploration, we will want the probability to have a spread out distribution and not let the probability distribution to collapse to a single value or a small region too soon. BIgger the spread of a distribution, higher the entropy H(x) of a distribution. Accordingly, the term fed into PyTorch/TensorFlow minimizer is:\n",
    "\n",
    "\n",
    "$$Loss(\\theta) = - J(\\theta) - H(\\pi_\\theta(a_t^i|s_t^i)) = - \\frac{1}{N} \\sum_{i=1}^{N} \\left[ \\sum_{t=1}^{T} \\left(  \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\sum_{t'=t}^{T} \\gamma^{t'-t} r(s_{t'}^i, a_{t'}^i) \\right) - \\beta \\sum_{a_i} \\pi_\\theta(a_t^i|s_t^i).\\log{ \\pi_\\theta(a_t^i|s_t^i)} \\right] $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import gym\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.signal import convolve, gaussian\n",
    "\n",
    "import os\n",
    "import io\n",
    "import base64\n",
    "import time\n",
    "import glob\n",
    "from IPython.display import HTML\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Environment - CartPole \n",
    "\n",
    "We can use the setup here to run on any environment which has state as a single vector and actions are discrete. We will build it on Cart Pole and they try to run this on many other environments like Atari games and others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_env(env_name, seed=None):\n",
    "    # remove time limit wrapper from environment\n",
    "    env = gym.make(env_name).unwrapped\n",
    "    if seed is not None:\n",
    "        env.seed(seed)\n",
    "    return env"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAASsklEQVR4nO3df6zd9X3f8efLxjFZDcLUF2T8Y7jUrDVRa6pbl4lpYklWXDbV5A8mR1rmP5CcP8iSaFU23Epr8oeVbGrIpGiJRBZUK0tDLBFqK8q2ul4ilKnDMSkQG+PiBgcuduxr5wdQGsc/3vvjfj1OzL2+x/eHLp97ng/p6HzP+3y/57zfCL/48vH3nJOqQpLUjgVz3YAk6coY3JLUGINbkhpjcEtSYwxuSWqMwS1JjZm14E6yMcnhJEeSPDhb7yNJgyazcR13koXA3wD/HBgBvgO8v6qem/E3k6QBM1tn3BuAI1X1/ar6OfAosGmW3kuSBspVs/S6K4CXex6PAL8z0c7Lli2rm2++eZZakaT2HD16lFOnTmW852YruMd7s19Yk0myFdgKsHr1avbv3z9LrUhSe4aHhyd8braWSkaAVT2PVwLHeneoqoerariqhoeGhmapDUmaf2YruL8DrE2yJsk7gM3A7ll6L0kaKLOyVFJV55J8CPhfwELgkao6OBvvJUmDZrbWuKmqbwDfmK3Xl6RB5ScnJakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1Zlo/XZbkKPAacB44V1XDSa4HvgrcDBwF/lVV/Xh6bUqSLpqJM+5/VlXrq2q4e/wgsLeq1gJ7u8eSpBkyG0slm4Ad3fYO4N5ZeA9JGljTDe4C/iLJU0m2drUbq+o4QHd/wzTfQ5LUY1pr3MCdVXUsyQ3AniTP93tgF/RbAVavXj3NNiRpcEzrjLuqjnX3J4HHgQ3AiSTLAbr7kxMc+3BVDVfV8NDQ0HTakKSBMuXgTvJLSa65uA38LnAA2A1s6XbbAuyabpOSpDdNZ6nkRuDxJBdf58+q6n8m+Q6wM8n9wEvAfdNvU5J00ZSDu6q+D/zmOPXTwHum05QkaWJ+clKSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqzKTBneSRJCeTHOipXZ9kT5IXuvulPc9tS3IkyeEkd89W45I0qPo54/5TYOMltQeBvVW1FtjbPSbJOmAzcFt3zOeSLJyxbiVJkwd3VT0B/OiS8iZgR7e9A7i3p/5oVZ2pqheBI8CGmWlVkgRTX+O+saqOA3T3N3T1FcDLPfuNdLW3SLI1yf4k+0dHR6fYhiQNnpn+y8mMU6vxdqyqh6tquKqGh4aGZrgNSZq/phrcJ5IsB+juT3b1EWBVz34rgWNTb0+SdKmpBvduYEu3vQXY1VPfnGRxkjXAWmDf9FqUJPW6arIdknwFuAtYlmQE+GPgU8DOJPcDLwH3AVTVwSQ7geeAc8ADVXV+lnqXpIE0aXBX1fsneOo9E+y/Hdg+naYkSRPzk5OS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhozaXAneSTJySQHemofT/JKkqe72z09z21LciTJ4SR3z1bjkjSo+jnj/lNg4zj1z1TV+u72DYAk64DNwG3dMZ9LsnCmmpUk9RHcVfUE8KM+X28T8GhVnamqF4EjwIZp9CdJusR01rg/lOTZbillaVdbAbzcs89IV3uLJFuT7E+yf3R0dBptSNJgmWpwfx64BVgPHAc+3dUzzr413gtU1cNVNVxVw0NDQ1NsQ5IGz5SCu6pOVNX5qroAfIE3l0NGgFU9u64Ejk2vRUlSrykFd5LlPQ/fB1y84mQ3sDnJ4iRrgLXAvum1KEnqddVkOyT5CnAXsCzJCPDHwF1J1jO2DHIU+CBAVR1MshN4DjgHPFBV52elc0kaUJMGd1W9f5zyFy+z/3Zg+3SakiRNzE9OSlJjDG5JaozBLUmNMbglqTEGtyQ1ZtKrSqRB8MbpEc797HUWX7OMxdcum+t2pMsyuCXg2P7d/PQHz7D42iEWXzv2FQxXL72JVf/4vjnuTHorg1vqcebVUc68OvalZ+fPnpnjbqTxucYtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTGTBneSVUm+meRQkoNJPtLVr0+yJ8kL3f3SnmO2JTmS5HCSu2dzAEkaNP2ccZ8D/qCqfh24A3ggyTrgQWBvVa0F9naP6Z7bDNwGbAQ+l2ThbDQvSYNo0uCuquNV9d1u+zXgELAC2ATs6HbbAdzbbW8CHq2qM1X1InAE2DDDfUvSwLqiNe4kNwO3A08CN1bVcRgLd+CGbrcVwMs9h410tUtfa2uS/Un2j46OTqF1SRpMfQd3kiXAY8BHq+rVy+06Tq3eUqh6uKqGq2p4aGio3zYkaeD1FdxJFjEW2l+uqq915RNJlnfPLwdOdvURYFXP4SuBYzPTriSpn6tKAnwROFRVD/U8tRvY0m1vAXb11DcnWZxkDbAW2DdzLUvSYOvnF3DuBD4AfC/J013tD4FPATuT3A+8BNwHUFUHk+wEnmPsipQHqur8TDcuSYNq0uCuqm8z/ro1wHsmOGY7sH0afUmSJuAnJyWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBrYF39u9f4+zf/fgt9X+wbPUcdCNNzuDWwDvz05O8ceqlt9Svv+W356AbaXIGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNaafHwteleSbSQ4lOZjkI13940leSfJ0d7un55htSY4kOZzk7tkcQJIGTT8/FnwO+IOq+m6Sa4CnkuzpnvtMVf1J785J1gGbgduAm4C/THKrPxgsSTNj0jPuqjpeVd/ttl8DDgErLnPIJuDRqjpTVS8CR4ANM9GsJOkK17iT3AzcDjzZlT6U5NkkjyRZ2tVWAC/3HDbC5YNeknQF+g7uJEuAx4CPVtWrwOeBW4D1wHHg0xd3HefwGuf1tibZn2T/6OjolfYtSQOrr+BOsoix0P5yVX0NoKpOVNX5qroAfIE3l0NGgFU9h68Ejl36mlX1cFUNV9Xw0NDQdGaQpIHSz1UlAb4IHKqqh3rqy3t2ex9woNveDWxOsjjJGmAtsG/mWpakwdbPVSV3Ah8Avpfk6a72h8D7k6xnbBnkKPBBgKo6mGQn8BxjV6Q84BUlkjRzJg3uqvo2469bf+Myx2wHtk+jL0nSBPzkpCQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmP6+VpXqTmvvPIKH/7wh7lw4cKk+668bhFb7lhKLvkOzG3btjHyk7OTHr9gwQI++9nPctNNN021XemKGNyal15//XV27drF+fOTfxX8b/zKjfybO+7l5+ev/v+1RQt+xhNPPMGz3z8x6fELFy7kk5/85LT6la6Ewa2BV4QfvLGO51/7bca+er749Wv2AX8+t41JEzC4NfB+enYZh179HS70/HF46Y1f48yFd85hV9LE/MtJDbxiAedr4S/UXj23jDfOXztHHUmX18+PBV+dZF+SZ5IcTPKJrn59kj1JXujul/Ycsy3JkSSHk9w9mwNI07UwZ3nHgjO/ULt+0XGuuepHc9SRdHn9nHGfAd5dVb8JrAc2JrkDeBDYW1Vrgb3dY5KsAzYDtwEbgc8lWTjeC0tvB4vrJNf97M85depFzv79D1ly1Y+5ZckzXJXJryiR5kI/PxZcwOvdw0XdrYBNwF1dfQfwLeA/dPVHq+oM8GKSI8AG4K8meo+zZ8/ywx/+cGoTSOM4deoUY//qTu7wy6f5t5/6NEX4tdXL+Eerf5n/A4yMvnpF73fttS6taOacPTvxiUNffznZnTE/Bfwq8F+r6skkN1bVcYCqOp7khm73FcD/7Tl8pKtN6PTp03zpS1/qpxWpL6Ojo30HN8CFKqA49IOTHPrBySt6rwsXLrBr1y6WLVt2hV1KEzt9+vSEz/UV3FV1Hlif5Drg8STvuszuGaf2lj9BSbYCWwFWr17Nxz72sX5akfpy+PBhHnroob6u456uBQsWcP/993PrrbfO+ntpcHz1q1+d8Lkruqqkqn7C2JLIRuBEkuUA3f3F05QRYFXPYSuBY+O81sNVNVxVw0NDQ1fShiQNtH6uKhnqzrRJ8k7gvcDzwG5gS7fbFmBXt70b2JxkcZI1wFpg3wz3LUkDq5+lkuXAjm6dewGws6q+nuSvgJ1J7gdeAu4DqKqDSXYCzwHngAe6pRZJ0gzo56qSZ4Hbx6mfBt4zwTHbge3T7k6S9BZ+clKSGmNwS1Jj/JIpzUtLlixh06ZNfX0f93QtWLCAJUuWzPr7SBcZ3JqXVqxYwWOPPTbXbUizwqUSSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktSYfn4s+Ook+5I8k+Rgkk909Y8neSXJ093tnp5jtiU5kuRwkrtncwBJGjT9fB/3GeDdVfV6kkXAt5P8j+65z1TVn/TunGQdsBm4DbgJ+Mskt/qDwZI0MyY9464xr3cPF3W3uswhm4BHq+pMVb0IHAE2TLtTSRLQ5xp3koVJngZOAnuq6snuqQ8leTbJI0mWdrUVwMs9h490NUnSDOgruKvqfFWtB1YCG5K8C/g8cAuwHjgOfLrbPeO9xKWFJFuT7E+yf3R0dAqtS9JguqKrSqrqJ8C3gI1VdaIL9AvAF3hzOWQEWNVz2Erg2Div9XBVDVfV8NDQ0FR6l6SB1M9VJUNJruu23wm8F3g+yfKe3d4HHOi2dwObkyxOsgZYC+yb0a4laYD1c1XJcmBHkoWMBf3Oqvp6ki8lWc/YMshR4IMAVXUwyU7gOeAc8IBXlEjSzJk0uKvqWeD2ceofuMwx24Ht02tNkjQePzkpSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5Iak6qa6x5IMgr8HXBqrnuZBctwrtbM19mcqy3/sKqGxnvibRHcAEn2V9XwXPcx05yrPfN1NueaP1wqkaTGGNyS1Ji3U3A/PNcNzBLnas98nc255om3zRq3JKk/b6czbklSH+Y8uJNsTHI4yZEkD851P1cqySNJTiY50FO7PsmeJC9090t7ntvWzXo4yd1z0/XkkqxK8s0kh5IcTPKRrt70bEmuTrIvyTPdXJ/o6k3PdVGShUn+OsnXu8fzZa6jSb6X5Okk+7vavJhtSqpqzm7AQuBvgV8B3gE8A6yby56mMMM/BX4LONBT+8/Ag932g8B/6rbXdTMuBtZ0sy+c6xkmmGs58Fvd9jXA33T9Nz0bEGBJt70IeBK4o/W5eub7d8CfAV+fL/8udv0eBZZdUpsXs03lNtdn3BuAI1X1/ar6OfAosGmOe7oiVfUE8KNLypuAHd32DuDenvqjVXWmql4EjjD2z+Btp6qOV9V3u+3XgEPAChqfrca83j1c1N2KxucCSLIS+BfAf+spNz/XZczn2S5rroN7BfByz+ORrta6G6vqOIwFIHBDV29y3iQ3A7czdnba/GzdcsLTwElgT1XNi7mA/wL8e+BCT20+zAVj/3H9iyRPJdna1ebLbFfsqjl+/4xTm8+XuTQ3b5IlwGPAR6vq1WS8EcZ2Haf2tpytqs4D65NcBzye5F2X2b2JuZL8S+BkVT2V5K5+Dhmn9rabq8edVXUsyQ3AniTPX2bf1ma7YnN9xj0CrOp5vBI4Nke9zKQTSZYDdPcnu3pT8yZZxFhof7mqvtaV58VsAFX1E+BbwEban+tO4PeTHGVsyfHdSf477c8FQFUd6+5PAo8ztvQxL2abirkO7u8Aa5OsSfIOYDOwe457mgm7gS3d9hZgV099c5LFSdYAa4F9c9DfpDJ2av1F4FBVPdTzVNOzJRnqzrRJ8k7gvcDzND5XVW2rqpVVdTNjf47+d1X9axqfCyDJLyW55uI28LvAAebBbFM21387CtzD2BULfwv80Vz3M4X+vwIcB84y9l/6+4FfBvYCL3T31/fs/0fdrIeB35vr/i8z1z9h7H8vnwWe7m73tD4b8BvAX3dzHQD+Y1dveq5LZryLN68qaX4uxq46e6a7HbyYE/Nhtqne/OSkJDVmrpdKJElXyOCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4Jakx/w+C/o15SKwP/gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "env_name = 'CartPole-v1'\n",
    "\n",
    "env = make_env(env_name)\n",
    "env.reset()\n",
    "plt.imshow(env.render(\"rgb_array\"))\n",
    "state_shape, n_actions = env.observation_space.shape, env.action_space.n\n",
    "state_dim = state_shape[0]\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Policy Network\n",
    "\n",
    "We will build a simple network that takes in state and produces logits for the action probabilities. We will keep network simple. he observation space and action space is as given below for CartPole\n",
    "\n",
    "    Observation:\n",
    "        Type: Box(4)\n",
    "        Num     Observation               Min                     Max\n",
    "        0       Cart Position             -4.8                    4.8\n",
    "        1       Cart Velocity             -Inf                    Inf\n",
    "        2       Pole Angle                -0.418 rad (-24 deg)    0.418 rad (24 deg)\n",
    "        3       Pole Angular Velocity     -Inf                    Inf\n",
    "    Actions:\n",
    "        Type: Discrete(2)\n",
    "        Num   Action\n",
    "        0     Push cart to the left\n",
    "        1     Push cart to the right\n",
    "        \n",
    "\n",
    "The model will be a simple one with 1 hidden layer with Relu activation and final layer being logits with dimension equal to number of actions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = nn.Sequential(\n",
    "            nn.Linear(state_dim,192),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(192,n_actions),\n",
    ")\n",
    "model = model.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict Action Probabilities\n",
    "\n",
    "We will use this function to generate the trajectory. It will not be used for doing back propagation. So we will use PyTorch `no_grad()` to avoid gradient calculations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_probs(states):\n",
    "    \"\"\"\n",
    "    params: states: [batch, state_dim]\n",
    "    returns: probs: [batch, n_actions]\n",
    "    \"\"\"\n",
    "    states = torch.tensor(states, device=device, dtype=torch.float32)\n",
    "    with torch.no_grad():\n",
    "        logits = model(states)\n",
    "    probs = nn.functional.softmax(logits, -1).detach().numpy()\n",
    "    return probs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Play game and generate Trajectory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_trajectory(env, n_steps=1000):\n",
    "    \"\"\"\n",
    "    Play a session and genrate a trajectory\n",
    "    returns: arrays of states, actions, rewards\n",
    "    \"\"\"\n",
    "    states, actions, rewards = [], [], []\n",
    "    \n",
    "    # initialize the environment\n",
    "    s = env.reset()\n",
    "    \n",
    "    #generate n_steps of trajectory:\n",
    "    for t in range(n_steps):\n",
    "        action_probs = predict_probs(np.array([s]))[0]\n",
    "        #sample action based on action_probs\n",
    "        a = np.random.choice(n_actions, p=action_probs)\n",
    "        next_state, r, done, _ = env.step(a)\n",
    "        \n",
    "        #update arrays\n",
    "        states.append(s)\n",
    "        actions.append(a)\n",
    "        rewards.append(r)\n",
    "        \n",
    "        s = next_state\n",
    "        if done:\n",
    "            break\n",
    "    \n",
    "    return states, actions, rewards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Rewards to Go\n",
    "\n",
    " $G(s_t) = \\sum_{t'=t}^{T} \\gamma^{t-t'} r(s_{t'}^i, a_{t'}^i)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_rewards_to_go(rewards, gamma=0.99):\n",
    "    \n",
    "    T = len(rewards) # total number of individual rewards\n",
    "    # empty array to return the rewards to go\n",
    "    rewards_to_go = [0]*T \n",
    "    rewards_to_go[T-1] = rewards[T-1]\n",
    "    \n",
    "    for i in range(T-2, -1, -1): #go from T-2 to 0\n",
    "        rewards_to_go[i] = gamma * rewards_to_go[i+1] + rewards[i]\n",
    "    \n",
    "    return rewards_to_go"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train on one trajectory\n",
    "\n",
    "We will calculate the loss and take a gradient step. We will use Adam Optimizer\n",
    "\n",
    "We are taking only one trajectory. so N=1. We will however, average it over the number of actions to get the average loss. So the function we will actually implement is as given below:\n",
    "\n",
    "$$Loss(\\theta) = - J(\\theta) - H(\\pi_\\theta(a_t|s_t)) = - \\frac{1}{T}  \\sum_{t=1}^{T} \\left( \\log{ \\pi_\\theta(a_t|s_t)} G(s_t) - \\beta \\sum_{a_i} \\pi_\\theta(a_t|s_t).\\log{ \\pi_\\theta(a_t|s_t)} \\right) $$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#init Optimizer\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\n",
    "\n",
    "def train_one_episode(states, actions, rewards, gamma=0.99, entropy_coef=1e-2):\n",
    "    \n",
    "    \n",
    "    # get rewards to go\n",
    "    rewards_to_go = get_rewards_to_go(rewards, gamma)\n",
    "\n",
    "    # convert numpy array to torch tensors\n",
    "    states = torch.tensor(states, device=device, dtype=torch.float)\n",
    "    actions = torch.tensor(actions, device=device, dtype=torch.long)\n",
    "    rewards_to_go = torch.tensor(rewards_to_go, device=device, dtype=torch.float)\n",
    "\n",
    "    # get action probabilities from states\n",
    "    logits = model(states)\n",
    "    probs = nn.functional.softmax(logits, -1)\n",
    "    log_probs = nn.functional.log_softmax(logits, -1)\n",
    "    \n",
    "    log_probs_for_actions = log_probs[range(len(actions)), actions]\n",
    "    \n",
    "    #Compute loss to be minized\n",
    "    J = torch.mean(log_probs_for_actions*rewards_to_go)\n",
    "    H = -(probs*log_probs).sum(-1).mean()\n",
    "    \n",
    "    loss = -(J+entropy_coef*H)\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    return np.sum(rewards) #to show progress on training\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean reward:30.374\n",
      "mean reward:46.121\n",
      "mean reward:56.162\n",
      "mean reward:56.313\n",
      "mean reward:160.657\n",
      "mean reward:157.222\n",
      "mean reward:170.182\n",
      "mean reward:105.556\n",
      "mean reward:206.141\n",
      "mean reward:277.889\n",
      "mean reward:138.455\n",
      "mean reward:133.465\n",
      "mean reward:289.808\n",
      "mean reward:447.414\n"
     ]
    }
   ],
   "source": [
    "total_rewards = []\n",
    "for i in range(10000):\n",
    "    states, actions, rewards = generate_trajectory(env)\n",
    "    reward = train_one_episode(states, actions, rewards)\n",
    "    total_rewards.append(reward)\n",
    "    if i != 0 and i % 100 == 0:\n",
    "        mean_reward = np.mean(total_rewards[-100:-1])\n",
    "        print(\"mean reward:%.3f\" % (mean_reward))\n",
    "        if mean_reward > 300:\n",
    "            break\n",
    "env.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let us record a video of trained agent**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_animation(env, save_dir):\n",
    "    try:\n",
    "        env = gym.wrappers.Monitor(\n",
    "            env, save_dir, video_callable=lambda id: True, force=True, mode='evaluation')\n",
    "    except gym.error.Error as e:\n",
    "        print(e)\n",
    "\n",
    "    if not os.path.exists(save_dir):\n",
    "        os.makedirs(save_dir)\n",
    "        \n",
    "    generate_trajectory(env)\n",
    "\n",
    "\n",
    "def display_animation(filepath):\n",
    "    video = io.open(filepath, 'r+b').read()\n",
    "    encoded = base64.b64encode(video)\n",
    "    return HTML(data='''<video alt=\"test\" controls>\n",
    "                <source src=\"data:video/mp4;base64,{0}\" type=\"video/mp4\" />\n",
    "                 </video>'''.format(encoded.decode('ascii')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<video alt=\"test\" controls>\n",
       "                <source src=\"data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAAc9JtZGF0AAACoQYF//+d3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE2MSAtIEguMjY0L01QRUctNCBBVkMgY29kZWMgLSBDb3B5bGVmdCAyMDAzLTIwMjAgLSBodHRwOi8vd3d3LnZpZGVvbGFuLm9yZy94MjY0Lmh0bWwgLSBvcHRpb25zOiBjYWJhYz0xIHJlZj0zIGRlYmxvY2s9MTowOjAgYW5hbHlzZT0weDM6MHgxMTMgbWU9aGV4IHN1Ym1lPTcgcHN5PTEgcHN5X3JkPTEuMDA6MC4wMCBtaXhlZF9yZWY9MSBtZV9yYW5nZT0xNiBjaHJvbWFfbWU9MSB0cmVsbGlzPTEgOHg4ZGN0PTEgY3FtPTAgZGVhZHpvbmU9MjEsMTEgZmFzdF9wc2tpcD0xIGNocm9tYV9xcF9vZmZzZXQ9LTIgdGhyZWFkcz0xMiBsb29rYWhlYWRfdGhyZWFkcz0yIHNsaWNlZF90aHJlYWRzPTAgbnI9MCBkZWNpbWF0ZT0xIGludGVybGFjZWQ9MCBibHVyYXlfY29tcGF0PTAgY29uc3RyYWluZWRfaW50cmE9MCBiZnJhbWVzPTMgYl9weXJhbWlkPTIgYl9hZGFwdD0xIGJfYmlhcz0wIGRpcmVjdD0xIHdlaWdodGI9MSBvcGVuX2dvcD0wIHdlaWdodHA9MiBrZXlpbnQ9MjUwIGtleWludF9taW49MjUgc2NlbmVjdXQ9NDAgaW50cmFfcmVmcmVzaD0wIHJjX2xvb2thaGVhZD00MCByYz1jcmYgbWJ0cmVlPTEgY3JmPTIzLjAgcWNvbXA9MC42MCBxcG1pbj0wIHFwbWF4PTY5IHFwc3RlcD00IGlwX3JhdGlvPTEuNDAgYXE9MToxLjAwAIAAAAG3ZYiEACv//vZzfAprRzOVLgV292aj5dCS5fsQYPrQB/gAAAMAAAMAHDTDp1EyExzlAAADABcQBHgsAjwlQthIR9jpDJwLM0QAZEf9/Qu8l5vwqb7zrTIj7lBqjwvaDgIB930Za/TQMHWsrKMLMC9BsXk0BJtwm4nCat8/3EIM95+7wrJ303wDNIgCEx6qPDJhKOEhPQeQBQQGjR2H1LTa2KPbuFQEDhvMWNxSk8YXLojEqxrfteklw2ESJrPkE1OAFOQKApheQq5z07pcRr7QiXkL9CUl4ZFF4Briz3HelIIqTZDRn/EAHZFg5Q2aoS7+5XWzmVbsj9N/Ly/t711LTvcrVe9ohqqcdJTaDXSb7CuNuKhexPgyCxJN5gaStrxvocl1qqwcUW6HDJQyGTganD+sSGuyYpz1sjLf1EYsoZ8OFdRwdP4bECCizqa4q/IWcBHXHiY9JQsw5ZHCgqmbppACgSr10iNyHXCwKNdxqQ4vkB/Um+85mpbH3PmczOmtFdmiAKZdJt6pOvgfMeo9Fqom8WfK8yOabgB3+SGBUlxFXNh8hCsVr0BwAHOTJQAAAwAAAwAT0QAAAFlBmiRsQn/98QAAAwKhkHGqAbLQb/TEt7VpDPtkPRO255uXQdWc533fHUbeejb30nVqz8RCMi+1x/DvwQXhkAp382RiBsiV5B2ORCStPsr4XjaLXRHzIQb0rgAAACNBnkJ4hH8AABa1Wv4WSVvqw2MxVvoylwKHNr7cimB20x9nwQAAADkBnmF0R/8AACPDQKqX4XBpgASMs9gkL25WjWz/E+7rTOl/Ek43PdPSxTcGnK4ra7HmH9e6cfD6C/wAAAAwAZ5jakf/AAAjxzpDLdABXtwt3LW4JPyqIXcAwQElMktfi9WJH1zKFrOtDY5hwhtpAAAAekGaaEmoQWiZTAhX//44QAABDVSwHoCETNhQV0eUk8gMOOY6hn9/On1CTHLb6M1JDzEET3rkutbt++j+LsWlQUbENBvd+1TvnEgXoD5xDOCjzP4CJnrim3bJPxv2U1TT39LxzggLmsUCkTsfPpriIn9/yP6+9Rs2LdyxAAAANUGehkURLCP/AAAWu4qUZI1jNYtpLAAfzWNAGWbQKsMhFp2tm7q5oOr2wrvo3D8SGoWCp77ZAAAAGwGepXRH/wAAIz3LOemD4BfYaDQWlzF4U1waEQAAADUBnqdqR/8AACOxzAGMoHcRNUD8mQPJJKmxXLbLTN8k0Js5ABO3aea7Hlv2n0qQvOjATBidsAAAAHVBmqxJqEFsmUwIT//98QAAAwKfAq02ToY8RvHE8pbqN6CdtAAhsF7ho4H/bATA3mdsbvOh9g3rMqNzIZvL99mI7kTGNqZ8W29w6a3Yf93zEGR786DxPUHWGBPlBN7dqjDi/gXPbRaBVCRCked75wG4jCxCCPIAAABNQZ7KRRUsI/8AABakbERAA4x+NV4JcORPTjdCP1HSf2BW9bEdEwAmP/+3NkLfVDw1JxArO4wkWqqC7ak8Z/UZ+q2MZ/0b5cEhJ+WMHJEAAAA1AZ7pdEf/AAAjRQm5ofF4n4YtU6/YAASmtcQSequWst0kAlulMplVp9yiuCBZ5E/Yq7W3lKgAAAA1AZ7rakf/AAAjscwr+XeA1y8M4/JZuztlZ0sgQxV7jVACVNplBSkU13DKGrDmk4u1Qpw5aj4AAABKQZrwSahBbJlMCF///oywAAAaoV0JJJNYpABuuDTjfdIfBOo1aPivVMLNOiNc5VUwns8e/2t8bNT47I3JzgnkXyCmDepV74husGEAAAAhQZ8ORRUsI/8AAAhvIeY8pfvz9nBRqjZU2lOLCqWqaF7NAAAALwGfLXRH/wAABRz2sVdXGvu4yenqkEUICcgAH87SHJnUwViEvHKdumWvprFvBgErAAAAGQGfL2pH/wAADYSVLQvzW5fCalhRKo4oaygAAAB2QZs0SahBbJlMCFf//jhAAAENaFWiowByi4GUBe5d+BuXxtMYVafQjV6vjdeTh9i+V8Aon2W328b5ECjEbox74g25qO3Bb+j0w/oV72ts2XfKhVXFPhz3B4PPGTCq3JcspO4nGj4n9L5pV8RqQdkXCJ52wBmimAAAAB5Bn1JFFSwj/wAAFrUrN2uJ8okfVKmG1hcb13wIOSEAAAAdAZ9xdEf/AAAjwJkyaycYJ16ivpar6FH4ayK3aJUAAAASAZ9zakf/AAADAAqGa2bB2oeAAAAAU0GbeEmoQWyZTAhH//3hAAAEFRcnK2zBQjQAPw068gbWOBg/R1cj1wU3fe3koecxPdYGOI2BiJGG8Bu7HUKEmYUjQnSDHkmmYWd7Od8LCbBkM6eBAAAAH0GflkUVLCP/AAAWvE8e0BDpUtjNHYZJqftpWCg+ICAAAAASAZ+1dEf/AAADAAqAuYuc/N+BAAAALAGft2pH/wAAI7IyOqUWWAFqwDA7wHLr6TsQ29IqfS43bYstXihQxgDowh+DAAAASkGbu0moQWyZTAhn//6eEAAARUUJ3IArVLi9HqUPMF7GEu/9ul+0m6lNLf9O664A6v9PoFK7uarpY1Th8d/rLj2xAi9ZGzRLn9aAAAAAMkGf2UUVLCP/AAAWvE5sasSgBaQmDnTDOjEsL5I0uEZLlouKzQkjo0/tb150QE+e5lkxAAAANAGf+mpH/wAAI78azkLyzBWTdQDzyownXquV2wwO8oAJ26GLmjXhNCFNvzTRjsV9xEHKSYAAAAAuQZv/SahBbJlMCF///oywAAAah1QQkEXPSbpvUlvwd4vZF4Qrjp9xHL2UjhVrFwAAAC1Bnh1FFSwj/wAAFqhldnPQAh0JpbWTXdTO2DCJMbrEafMrSanlDp7yOweZbekAAAAoAZ48dEf/AAAjrDxUVPv+OUIZABOwDPkNCF6R9kzJZXWDEa6gD+G/swAAABABnj5qR/8AACOyMKMj1eIOAAAAI0GaI0moQWyZTAhf//6MsAAARlz4LBNmCACc7fdVFSjJD9t7AAAAE0GeQUUVLCP/AAAWvE7geuAmoOAAAAAOAZ5gdEf/AAADAAADAakAAAAQAZ5iakf/AAAjsjCjI9XiDgAAAG1BmmZJqEFsmUwIX//+jLAAAEYQ/3/AEHK1tCQSpR/uCkWKDuWU4axUfhhCfYYl94r3FHdFcdrU6/AeY64AmeSh5cKGFrnG0aKaeQyLjqlOStSNzwA7nHfSGsvvt4/umHN4sXKXZtYw45jpdT09AAAAIkGehEUVLCP/AAAWwJ9x2kpf1RZ1oD58Ida6/aYFydrg1YEAAAA1AZ6lakf/AAAjo6jNl++tdqs92G8bqJ2F6D8VBbubS655kAB711bMESL9/YHDmQp6UyXfHbMAAABoQZqqSahBbJlMCF///oywAAAaiUe3alrbx5gS8UBPUef+r7pBHXrHR1tvNiWURIOjvsKf3SHULL11KnSv6OYRYcUD3NDCUS96T6BnWkjEv57TM/P+CYD3/PBA6HlTSc4kJ138Wk4pM58AAAAoQZ7IRRUsI/8AAAhtfENExtOMqLrgQBjZ8lCroxXAolvKcIWd2JYeRgAAACcBnud0R/8AAA1cwuVfGAoo32vpUfwvy+5xuDCzbneee65piMbGGzAAAAAbAZ7pakf/AAANhY70YpTiM0oCghlXSk6MTSvhAAAAf0Ga7EmoQWyZTBRMM//+nhAAAEVFCtJACtUfTQqHEzwhqIsQfPDPzCmcutJFt/cDoL3Sm5cDCB7dtdASVoZzKO6RI89/FMreHhiVehFKnleulQCnzv4XG6hc29ZnekistqxWWXoGmZyGKZIJbrGk7fx7MG3S/+giDEJUEES24oAAAAAoAZ8Lakf/AAAjo2gUDN6KaHG9jJNIvWq5mwatV/yAc0pVW2d/cogSoAAAAGdBmxBJ4QpSZTAhn/6eEAAARW1ne4egCE+HzFL4HrikayP2S0IlqI5sQOXqItfXI4+PPBhwooqXHymjm7zwu/cy+Qd9siR0Zi8c81Qc3xCm64Eb9L0dus6CvNoyj1+jhxKKcfwjEf0ZAAAAJEGfLkU0TCP/AAAWvE74IDnfFUpEEwUkNcgjHmZGLFZ7RCbyQQAAACgBn010R/8AAAMAugpNNqS5diIxcAATV0MXNGvCaEKROkj89fmz23tnAAAAJgGfT2pH/wAAI7IxNv3bsUQR7J20SYAH80vk0sM+jZAzg6K6x5yQAAAAT0GbVEmoQWiZTAhf//6MsAAARge5ABfa6Do0jYBL+eXH/H6eyM4EVa7FA/JBtsKthE2u7aLxfBQMhQSr20zdlKaXRq6vuuYmyPdzYIDqsLgAAAAeQZ9yRREsI/8AABa8RxaEegDm8w9NblqjsFjoEnBxAAAAFQGfkXRH/wAAAwC6eOH/1njFcsaaCAAAABUBn5NqR/8AACO/HAhfv5MpslpXQuAAAABQQZuYSahBbJlMCF///oywAABGLwI4u4A5RnjtJpZgGB1VN9s29i1CNCgNg6xq7zIHa/Xt8MTaE96w08kQzrP16YTRsytA19xAVvzjsT8EGs0AAAAgQZ+2RRUsI/8AABbAmz/M0osaHaTxf2Q5gxQMYjpCckAAAAAXAZ/VdEf/AAAjrDydNLVhzZuMoW0KcHEAAAAnAZ/Xakf/AAAjx8cm3xIZxi4ASptMoK6pGuf11AjMIWAltTaYWb2ZAAAAJEGb3EmoQWyZTAhX//44QAABDT9ZJGu60L9xCcwAARC69MJfkAAAACFBn/pFFSwj/wAAFqUAp4AbmiEsHXzcuNnHGYG2ydtqQwcAAAAVAZ4ZdEf/AAAjwwOgfctiZt8bZoXAAAAAFQGeG2pH/wAAI8e3qH3bBE037paFwQAAAFlBmh1JqEFsmUwIX//+jLAAAEYoiTYBqVqQUaRsQcxq6+YxL7ZsjtJcSIxkyuCt9R3VSpvCrICu39LLEACjqmgu2V9RspR0PpqiChKCEXCLd+6X8TKTdDYVgQAAAG5BmiFJ4QpSZTAhX/44QAABDPo80ALRMkVqErhTSlZgib/C+IxA2qJ5Yc+DYmEYRlL9pxLpe6uynMIcOWmjEd6hAjdCywMh7iVZE3nbOYsrUGq38HSl9P9mhNzknq50fROoQHDgHuRz/QoVa48KYAAAADVBnl9FNEwj/wAAFrVYrdA6yWfrV8266c4ltX4yIjyAU8dWABDXTWdiHpU6A5/ab40xDoRW3AAAACMBnn50R/8AACPDPggO04OhRq7RUULQEg1jqMqlVf6gVBf2zQAAACABnmBqR/8AAA1WInjMgCqCSSQWfrM7HuVAXgHteDxD5gAAAIZBmmVJqEFomUwIT//98QAAAwKfAy28Ao5W+jL7DEIp/9qMiHn1fVbD0p6FiQSqTzq1yMHaWB2z+ncQoSEhUmygvhYBqMi/JUc08e/cUV/Db6o5vo4XBFbXSPsBNYJvIp1cFnMbuNdK7QZQipOFsYruQ8Kifd5AaKry2CrgYbBjaJt8Pw4FmQAAADZBnoNFESwj/wAAFrxOQvz8D7D3ivj/vqh8vC+n8Fpyempn9ai0ImFb85UdpFAFIupevqmGbZgAAAAkAZ6idEf/AAANf0is66PVlJpYxeWzMKO/B5wAfuAcFbMUh9mBAAAAOwGepGpH/wAAI7IsiA+Z+5ldO510wWg0mxbepK0/zwAEPlPuZj5NiXUOS6Il0NU/9T251bcDmugXV2DBAAAAXkGaqUmoQWyZTAhP//3xAAADAPgTW04AR2n5te7ZaPx601ctCP+sDNLvNMRhcHZW6Q613jrnRxncFMtflkzbpdPwonkfGNJ4i2u4uzcqXu7cnQuu+hkRljFWxS1ZRcUAAAA9QZ7HRRUsI/8AAAhnbJvDkhxJEX30M5yARi/lumVqMI/Er8tT4Ad9BjQAtwc0pJc9fg0fDFfkjPmQdx+lfQAAADsBnuZ0R/8AAA1/P2b9kJSYyF58Pf2h0/5NKfr1k+R2ZyYi+nM1w+ACVFGz/q0H7BU+LdcpWSfLE+ehAQAAACoBnuhqR/8AAA2FkcFmWhWYYfurUhWpTdFcw65MnogAAH8GluJXzc9oJQQAAACBQZrtSahBbJlMCE///fEAAAMCnwLEmLoRoAEJVYJONvZDVLfuu3+XVO7eG8suFJeFh0tdfcrF/vNfGeG/1i04j3Cpgve9UiVKgxUm9OoQQdwtcTjCv+egze9qQqkjq3BIJuGbbqkvCvRV93bb2uGVTXj1kZc97fXSS8RmBCSpoLvZAAAANEGfC0UVLCP/AAAWtVmJ5eJ0GuX56bAfiMATuHFh2hJoWwOPqjg9mMrDj8BW4kuQOU3zbMAAAAApAZ8qdEf/AAAjw0cbnPH9GMxNV+xZwRs9p4hwuIIgCv3hCFlFgSa9y24AAAA0AZ8sakf/AAAN146KXtAiMfJCm8SDuTDc6FruWaJIuwKhqjcI/OQAcVJ//Sln3cAzPBtswQAAAFdBmzFJqEFsmUwIR//94QAABBNb0bbCNHIACOtHGQSyERv+5JErOp7uTeN4OEXs6VwiI/Mo1ILniRQ4T4TuTcS7+0eg5Pe2oV0Wmb4QaD7RafRJld569i0AAABEQZ9PRRUsI/8AABa7itHptd6hrfdR3svC2vkz6275ZBjOk5eRmEerhYLYnqagACcYpYeHxLdGyfRQHiN5C2/6Rr0KO3EAAAArAZ9udEf/AAAjrEJbrUJuIixfz4UuTic29MyM/tcRv85p497SEYyUJ/GLZgAAACcBn3BqR/8AAA3PqGX8Tz0xXiCdjaOYIqHTyl9FPzwVnHbA7FUstuAAAABeQZt1SahBbJlMCEf//eEAAAQzW/Xg1xYXTolMHGNfllFLw+evLMozhdJwrHoD86xzsRSRJCNbq4le2QbWPniKt3OoinrbZeKXkzpmdsQtd7waujQSDcvpcFe19U1ZeQAAADRBn5NFFSwj/wAAF0NHtwyUfCjnnQ3X1MJvU4kzN/XsRhWmwKsZFI2hS/M0JkEvTc8QLDbMAAAANgGfsnRH/wAAJMMXaI7VQj1oWqyF9O//Zu9mzPA55RjXm0AH17YS7hGdHkFJpl/GZBI47Q+2YAAAACgBn7RqR/8AACS+qhZ5Dv/kR/QSnq6USi7EaqkALWxE/A1+PdZmY1YXAAAAa0GbuUmoQWyZTAhH//3hAAAENo465BE9S8v7bIKm3AcAqppUTN4dVzqrtHlSGrNF4YDEkpemcVo+ma99TCGzsDqiS78verh8qGbFuJo8R2Eyv4y2x0orNNyyWAnuOnG8ihRYTUCNmfzHecPAAAAAP0Gf10UVLCP/AAAXQ0f7cZBX+FYwKBlMnq7RAmSAAbqJQ56wg78fAk7CfnPt/CYOUcrbz6XELZvXSKyGRvoCVQAAAC4Bn/Z0R/8AACSr7/kBZmhW4o2qDMdcuzUocLd/yHIExOlR7A5V5MWAgD6fgi2zAAAANwGf+GpH/wAAI72rdD+zL2Vx+yDhoUY0McFqHUuldy07XFKMkEezq3euQUhPgBJ9cCsy8zKj/bAAAABgQZv9SahBbJlMCE///fEAAAMCstQWoGv1gKCVvisAZ91hz41287IZOUJoW0PyuAPyXYE1+Rox5Gad3vYXQbAtemNqGdrorj3v5o/SXgAcRovsLxWxZ+gdELaYL6Sj7Zs/AAAARkGeG0UVLCP/AAAXQ0e3DkZszz+ClRZs01A6jFFD1WZrdwo+epQqGZPcCEqf7YvdR+NbEagznkrF856o8RNtwcfYtJyWWpAAAAA1AZ46dEf/AAAkwuzhLFLQALoeL5v/AGD1OBAQWNXqeUgldpW7P6Xa9VdvnA6AZ/jJ3mS0j48AAAAvAZ48akf/AAAkvwQf/f0x8DKzm6LqzY45HGOoPamMpXCDZCVoV9gDN8QDOEwi14EAAABNQZohSahBbJlMCE///fEAAAMCstKRD2ibPgOsJPClYLZLkuJkD9ncGegAap/Mr8N09NotL8TT+/OcD+2AcABceNqA04X2fjx359z5Al4AAABWQZ5fRRUsI/8AABc9NQcod6KDAJnsxKHGN/rFMrervgC185vZ5eJWaQyPsjqkBfDbMqoJsIHd//tlqzlvGsomDlEzxABOvHJGf++9raioXfQ11QLD9rwAAABDAZ5+dEf/AAAkwJkyc1yirdP+KzFyYALbqt/TcnMvf0/yS/z02soav5hBAhHlwQEZ1wuOxMLD560408GULzWWMGK1IQAAACwBnmBqR/8AACS9g0Jl4DeMer2hNhSjHpALYxxy8troqK7EXUVNlekEujJakAAAAHFBmmVJqEFsmUwIR//94QAABDM/o6GBmn+7yh8isy3iz8LQOODoZ2bLF/Q3AB+GnX0YYFj7U9nOjNeGM2HS7LPe6KkNvxNVt91CuERGuQjR3qonLG3IThJCttElPAejejPT0nqklgFolDIOSwGiyYM9wQAAADxBnoNFFSwj/wAAF0K2zaUAFwX0RNr2uk626xvrvynedjubx/sGOKSBh4pOT+aOjKinKeHvtchB4Y5i+vcAAAAgAZ6idEf/AAAkwJnd62QkO9gG0sgykGMZIe0MKrVtq18AAAAdAZ6kakf/AAAkxwgUye9zvnL/sy1IqT1X2fndn+EAAABcQZqoSahBbJlMCEf//eEAAAQ1FEaOIAbfebV2VpO9nq6uJ8cBQ/5LGi4zwibcwAEv9fJ1ZQ6e6DkxUVkCZLleKMgb/7ESvxiQty16/ujSz4bjV3k/zlUzYds60SEAAAAoQZ7GRRUsI/8AABdLis/4hNg53nbE4FPy8zQnBY3VPW/obMuUjyjUgQAAAC0BnudqR/8AACSuX/s46PxXUO3Aj6fXhU4AW7+JRgF7tnbsh0iKITkbB0kG+RYAAABfQZrsSahBbJlMCE///fEAAAMCwujHw80b8kkBM0Inhixd053MzqQi5BzEfgIjSGi/Hrt9KfKEf2azzu2XqDJgUKfbXG4hx/2FI1X2XJz74Bqt8MbgWjyqh8IMRtwgVLMAAAAsQZ8KRRUsI/8AABfphsXUvsHjQebN3eRb1lEzFCiGEnWQMZXtzOrIGnVowIEAAAApAZ8pdEf/AAAkwJqSDmfzf+Au+bOAHUKuUm4YmiCTsIOvnfvD2DygugsAAABCAZ8rakf/AAAlvwsJKZgAFrPbDFut0fzFQGtlU5xHIN75iWM+5B5Ij9iHOSiG5T2AZidupnoze8uSX3YHrpbsDKQYAAAAVUGbMEmoQWyZTAhP//3xAAADAsa8BiC8UdAAnQ06PEMHxnryRkFndQzZ3PAn3ZUStH7jB6kSC7bHpkZXp/0P7XWj4E47P5Msx/ArN4LLlle2SOIzV4EAAAAjQZ9ORRUsI/8AABff6+i0PNPBi+bCp67pwcY779yDBnv/IMEAAAA0AZ9tdEf/AAAlyGE3OXoAHE3yMd0FfHTSL2/MHlGArVUFo6/cFZ8AJW6vz2HKLla+XzVSYQAAACEBn29qR/8AACWuXtC0PnIFkO+9p0J6EwTWvrkDY48U24AAAABKQZt0SahBbJlMCEf//eEAAARXM5YD/y469mP7R0AAmR6DhbilO9mGJv8/RCt8rS0jrfa1qXYTLHGVgkID37DQKVxlOgD6iTKbHqAAAAA3QZ+SRRUsI/8AABff6+i0PNPSqn8SnCjIZ8CrNagnMSyRetFrFgAmR5Io1RLQreahZlCYAkt04QAAACoBn7F0R/8AACXAmR/PZYS0aV+u75EwYAJY/NWhrRg48y+g57cfdXjw29IAAAAyAZ+zakf/AAAlrl4zgXlFHbGAAtur52arL/UsxyVrd3Ko+MpJLZvoJ536MgkBetqndYAAAABkQZu4SahBbJlMCEf//eEAAARTSGYoA2kFW13ZMLYkN20dk/5jt906uyyQ29+68Y++hCKbWyQZ9gYtw/2isoY9d90hOypJ2JTCQH1wl8lQJSjJIhNR2OA0L3mIFXfBxWanyVLCSwAAAEBBn9ZFFSwj/wAAF+ibZgRvM6y3wwON8lbanIKy+ACcYldJFADDhc+2JbVBn+fhHpnwpwSToy4Dvo7639FAkL2fAAAANQGf9XRH/wAAJcCZ3ef5TzCh5tbLmUr6pRrU2ILABI6twijStGVBheNeQX3QITEF/Gw2mUnBAAAASAGf92pH/wAAJa5f7uZGXswjagMft0gAXUAR/Tb+L+qm4nj/OkLAstm2jj6zhGO+EDYc9GXuiDgQGj7dBlAhqM8z+DAtJbbPgQAAAF1Bm/pJqEFsmUwUTCP//eEAAARUsm+wANIrHV1yegYxhbdQQhj4/kl9OBdO+2q405kbTG5aJ1bLvK2jLGaN8HP0Qdlhwv/to+xP0b/7EKK8mHlBmIV5mBU/kQJzQ0QAAAAnAZ4Zakf/AAAlvYQEBPL/6AJzcL5qnRpE9N1r8hOr+2uTJpFdQ7OnAAAASUGaHknhClJlMCE//fEAAAMC2cn2CWfmtZIt3sih3okAJVd4y5V84BQ0aQn7tErS0I7TLAZhKeU1MmqX1hLAhgVqPLJwPk9OQ0AAAAAwQZ48RTRMI/8AABff7I21JmmK8pAiwTREugetqy1ePpx86wyROgG2tl1tyvJpOuCDAAAAIwGeW3RH/wAAJcCaluQsrA9ycaptz6d8kXsvkK9Fjh0pwLaBAAAAJwGeXWpH/wAAJa5f+zgRif13xhclxf6QHnig2GMkdN6jsAtPIitlQAAAADFBmkJJqEFomUwIT//98QAAAwLYbcMYnqEWHfgKucfu/QFTHEPhX0uDfS4U42WEQUb0AAAAPEGeYEURLCP/AAAYf/bHRqGmqAIkCTJgKG/XkNWyJwex3N48H3gPXvjNFIkztH/PBnFezk41S/chqbSRlQAAACIBnp90R/8AACbAmd3pXpFlEfqB0DMW6qhr59t4HFVXw/lQAAAAIQGegWpH/wAAJr2DgcoA+ycE7kb3IF4ZAz+TCwiAhFR7gQAAAEZBmoZJqEFsmUwIT//98QAAAwLYqihf6AgFDez1YJ3jsBSKedJh3OYT49XkHcnt/0k31Y+WNMxJZ75rRZl6zonrFKgBfGsKAAAANEGepEUVLCP/AAAYiJtupuYO77h3OlyGxCB4gAcWO2MBgP/Gv317+1N84P+iZXE4QC1N3CEAAAAfAZ7DdEf/AAAmwJnd6V6RZRH6gdAzFul+n8VfMA55UQAAACMBnsVqR/8AACa9g4HKAPsnBO5G9yBoy+VBKWYzqlOeoUZ7hQAAAGpBmspJqEFsmUwIT//98QAAAwLU6OCgAT+FoOvLOtNSbcWC/amhEuECB1ZjdqyUSw85owcYpXA2qcLsCdRYG4uohxxHoeCvULDz9IrNWyey6szczEQGt8r+pk0ozVCiTUvWr0/WS3e0LkQhAAAAP0Ge6EUVLCP/AAAYjXbh0joIE1cc8SB0oW5/OkhQxPIgjEkalolQABDcUmN+rNrckgrCRdee6q40fxHjM1P3CAAAAB8Bnwd0R/8AACbAmd3pXpFlEfqB0DMW6JmON9a3oO7hAAAALQGfCWpH/wAAJjuBaCADiwYqSpKytXBxjnf7kcN+uLN96BwOlAt9JdaKA5SyoQAAAGpBmw5JqEFsmUwIT//98QAAAwLV7UHb3AJrk1OxlXMHQVjg2BQ5rQv3N139Yucs7S3RYvKtjpnIJfsTAtpybXvA2J1jY12kua+vHE+pjgk4MCm8Xqw9cnHeBfIoNqbpxpTQa9USwxj/HNPEAAAAQEGfLEUVLCP/AAAYiJuEym0S7ls7zd1wFvwR/YqVpFhVMAN7ABOsIx3AQX0aerl4pw9B8oQfJJd6uG4ppVCwPcAAAAAlAZ9LdEf/AAAmwJqhnpm+q6bSNkH9I4Nrvn0ZlO5gyfBjahpjQwAAADMBn01qR/8AACa9hAP8l0CbsaAQU8OyaL0GBCPlXAAEr53jFCaqdhPnXBbUnhkFr3KysdcAAABGQZtSSahBbJlMCE///fEAAAMC5u0an0BrwDWtmeRjAJx15QB/9sSbV9F7pT5oya3D9GZqPOiC2pZa/4HJSZDYhjFTpFIz8QAAADlBn3BFFSwj/wAAGSl8w12l0fc+h0/HKQYOtsr+UpA69sy3Q9t4F97azvpawAEqKggq1lyt3Y7m7FgAAAAlAZ+PdEf/AAAmwJrMalPzKq4I/C5HQgVjlaf+up6AYLG88eR1gAAAACkBn5FqR/8AACfFU9M84lIxa+sYEtV2tSZSs8h1vvlqklRvEdDsGOdtiwAAAHRBm5ZJqEFsmUwIT//98QAAAwLm7BygAotzvXwLZhLYZnKPgAxXXVALZDhTjEPh17WlxyJVQkAdXaNTWHirFMTwGnWMrcyexHpKtdExuMB+0hYtQGkV1ZWGNRIIcGl+TMsgavKeEfiU7QB4rOVCAR1L831JgAAAAD9Bn7RFFSwj/wAAGSibbqZIopFX6QwkupAApCJDuSEF/9I/5JSTTAA4p+jyDLmpqUuvRG2SYQG09saaUiva7FgAAAAqAZ/TdEf/AAAn21cl/jjTXsISLQGJ/tlQ9/rGlXBDhlKWAF34cuRpgVsXAAAAKAGf1WpH/wAAJ9eohZN+bMIBN1+5++eaX0/aetlGyHKI5TbsMUCTuxYAAAB5QZvaSahBbJlMCE///fEAAAMC57h8dmGl/24Y8tYpMKGhWQCVmixPzIF3X7Wn2qyCoAsOa4+T0Uhi43eGU+S02BZglsypMr1yAEYeodr5+tiMES9EAWIFDcYejOxJNns3TPd/9BTOpjK76hnnCqCDiEk2Mf+HoT104QAAADdBn/hFFSwj/wAAGR/r6LQ808vSGLrDw8tLjkvaou8RcFDMUH3Ftwi06v7odyC0T325YT4rLq1RAAAAOwGeF3RH/wAAJ8GNhACLUNrQmjkR2/vKttZmtOuY4TLtIl1sLnJxc7ayKuRNsb8HYBuKnylYlB/waX2LAAAAOQGeGWpH/wAAJ8VT0zx6gdr5iDpQhJR/A4UdUvqf8UwkqPUYI0Dw9NFTMLAqWeFBAh5gow0AylGusQAAAIhBmh5JqEFsmUwIT//98QAAAwLorHKO9jwDVf1IWIuBfxhU5HIcFdes6MUyvlMOrto5/LkC3iWa6sElwFAR0jAq/RmSvU+rWPjNSGEsw5M8ebJ0Im/icahZ3MJJIKIqieIhSJoOIbxrEMuo0GLbVWpkmOj54Rv7z4XrdFLEMjGdtAUkLc536nMmAAAAREGePEUVLCP/AAAZD+Xvs2EmiCSANwUYY26J8NBm2QVg0eOZbCxcfdUkpG+ooeiQXYic7FXLmoweWXIwcMU7OF9VpA6xAAAASgGeW3RH/wAAJ8BY93HwNABasAu8GULRCuJ6QCW6UymVWfM3XyrGbZ8bhvp3uBuPLhpFaOd1SW9IjvqVezOa0/jyDSK5+lWm79uBAAAAPwGeXWpH/wAAJ6gj32XRmQg4AF05vf3s8cLG7fg9rnLxYSVsoE8XvHkGF4L9L8HqtY7vipQMZxtJtI8SYY6OEAAAAGZBmkJJqEFsmUwIT//98QAAAwLppjIs8pbhAKSdR55WnLgDIIPVga+ZrKasLDg5vVKdZyimbABjTBsvXSBNOsDmpxsJTbZtluoC5Hb8oFQO4u88c0igtgIGSHOo1kT3eBmCl+2xqJAAAABZQZ5gRRUsI/8AABkaR8LEYLWm45l3V3yli1nESSRES0//aw/Jt0ZOvmAjDhE3DY2Gu8ejBC1d8ID2qgA+6NNSWwqFtUoQWg18YSd1nKSYj/tdiCuEsjYF7LEAAAA2AZ6fdEf/AAAnvK8OBZ3MUXT+6ewwHfIL78/tRUbgsiw11VJW3eCHzvHanEbL0hPZJwkKR7HgAAAAPQGegWpH/wAAJ8VUnGNtPg14XV3RTPVmDTxJqAiDQ/n5lFVocucZuJ4SzCdgLl7gaPI+AEn16alwUBWZ2BEAAABtQZqGSahBbJlMCEf//eEAAASWNaiR6AGU3tbs/VPaIjL3AwZqCxY55YTcY9vospnb9kRk4vc3juN1TeQTyhwHki69Sy5xb8gaCauflI1OpOAVEB5Wi0Xt/rms+jC7mswfBfazFYllHCc/C/s4WAAAAFFBnqRFFSwj/wAAGSCIj3zBwYVpi2jYOzly272+T9/Ys5P7kQhLek4ftcorX+vKT5Bf3KiAFkk3WP8S1/9HOr/OBpS7pEW2yA0pWTYTiDIM48EAAAA/AZ7DdEf/AAAn3kuabc+4kSlb7DfSFN+9fZQ5txxQlb2pWMgcMQ1LMQZKACDqcClKWW5oao5hcbliG71FA/stAAAAKwGexWpH/wAAJ8mAf1BTXqy4kmDR9ZH6rvmGsMd3+Afo3NA0V7AkIQFORF0AAACzQZrKSahBbJlMCE///fEAAAMC6bO2bW/oBSTqeEjEUZDeiGdeYTn/cYbmd4l76tZgULSDvKmsCwoFNI4Wg1HeZlIz40fd1rZJeN+eu52LDI4/FvFz2PhekZgfoMK/X6b3ECih32Hp37hVnBgIMkLPwCpKvG+2wF4f77oapqtZU8FF3LuIp0YXYAClMWoaf1n9YcIOLf0wcE7Pei6k0LHE73zFRsfyDfMeK2IvzBDXYzS5hSkAAABLQZ7oRRUsI/8AABkrqeJaVR/6e0xhwheGU9DLZ+vjJo1tVy3uY83UIOGTDottghC3UTLTnPJTkmkYU3DKQMoPCDahNYq1QfHoFrlAAAAAOwGfB3RH/wAAJ8LnIDl39wzxt6kUQRBOCldBy8G1A1lvE9/LAie8kllz8Iia/75vgbWoG01QXDtgMlx4AAAAPQGfCWpH/wAADsynMP+G/t+cvCQH6ubFEVVH3XuIqkjnQaKmhgKX8vCrRl2mIje+5VnrwHEmXAkb3yisxgUAAACaQZsOSahBbJlMCE///fEAAAMC6QXrNwFBFFXeBAjdSNtS147QmOfsK8kySSUHwXGdmS1IU41cGPo6jlQuR6dvKE+XQve+Ga4ZQo+ZseInYafhjhY8Uic2IaYmdjtcroUkS3M/P1oIh1gQS2VfSaP4THL/uOeFJQuO1BS+Y5rG+n7WYPN4VDOb6D6/9X5L/K1H7+Qut3rAICaBgAAAAElBnyxFFSwj/wAAGSKfSiDnXHTKmYxyiPN5VTrUE7gd01VV+ZSHdECdSNY8z42AEzpeCt182pgK98GjCPYoJc1mdjlROyanN31+AAAAOAGfS3RH/wAADzV08xmloBlT8Z8fBUVaEAohu4P2NitQKpaTJ/GCBzMSoUdf4sQAldDENg9cMcOBAAAANQGfTWpH/wAADy5X0eOQjn1MbtxPYipA95Ek3VfeRJ7ThMSgAnK3lSzwF1d4hpVLQdiGV2HBAAAAd0GbUkmoQWyZTAhP//3xAAADAudP3QBt+KJo1SwA98DBAtO6Vzdz+Zd3FQXiIW3vvWcVNjbwU8622ZeI5RDwEJVB+BycyyArHuUh1tvgjdC2GIxS//SLJI7OhbTI0/ElIAypEMAf4DrDHERs4MyMmOLU9wuZl8uBAAAAN0GfcEUVLCP/AAAZKZC29wxD7OlsnS7KyymvYfqBrGfT4DEXa2jD6uXpW5lcuLeAAXQAtTBniw4AAAApAZ+PdEf/AAAPN8WRNVeCTUqNbidAUCzmt+BhyeZWGBko35ti9buNr4AAAAAxAZ+Rakf/AAAnygS/l1BUb1KuM5MlieFF+yJ8K/TD2+AArqQk27g2DyPeYkGHtE3FhwAAAK9Bm5ZJqEFsmUwIR//94QAABJUX09LAFlBG9OZ4oLHSGnrewRM8OqPJ8NZXe5e5iA8AcSaH6e8i1jzN9P2aFBBeWFO+CPIuJiFJhkNJoe5krBNtOakrP928Rcld/+xjW62VPF042LCLqL7ytGeD5Ybma81WHwgF+Fz/+eKAJ90VeTkX6yR0hSYF34Q8B555ANNr6mPKO604rn8Hw82r/u/9VCxqOcHI2PxpESkBgK5bAAAAQUGftEUVLCP/AAAY+GPu9jUShtczjwX+0kYNqHaZOGsPfqTl4VROTO7Na0i9ECQjsGCNEDwbxmGqAD4aIcW8PjZ8AAAANQGf03RH/wAAKP6D/iSUq7BDz1oqpymwzsQwBNkTuCpxZJK4AH7/3Aqvh8C/9NRT8/lKETAhAAAAPAGf1WpH/wAAKQP+Tfx0IvZBCjDpu6XswlhU9z5i9cQ7YEtgAnbxMTAN9Tqhu3QRuM35mPArG9Z2RhWceAAAAIhBm9pJqEFsmUwIT//98QAAAwLpDz2ie+AOMBdwUaFaBWxVMnmLLdpzsG8uDyEQF3WhzVeDeraR2RQWeZ5HricHTQcZv9peaxs/8DMp5mOOECc8UrxHw0DJGVHpzLsns1LQyAnW4LWi4mVRH8smRNuPc9AlvuLMqEgH4ZMKOduRedPnOY14XI+BAAAAR0Gf+EUVLCP/AAAZKNSR0qiQYAq6ZgItiPpzXT2qjh7k5CSo/8jJfP44RAC3HtYKmIBmaRxXfvHgpjN1afTZdH/tZWVl72WBAAAASQGeF3RH/wAAJzsb7eJqvEkFk0RLAFROuciCatgKlWj4Z/bPYW4KAAPviE5lbbkYFBWoSVThbkWq6AUECejgpQlVBxTw3kX/ZYAAAAA0AZ4Zakf/AAAnxR+FzkGLRE5Cvn9T0mn0KQlativHtlSRU4zjP0YoZYiFPf5n1zif+Ub29QAAAGxBmh5JqEFsmUwIR//94QAABJY/XuM0gFo6FrNevlt5yCAGf6CZwv5aDFs3NBOidEQ50GXiAqSp1PwXaAkPrljvleKEuG+VaRpNYJbdc2q49I1rX9vtCzE2+ixBDC2x151bMRgsy1bN2b2ofuAAAAA5QZ48RRUsI/8AABjTnFP40AUZOeccNakMLAqP+oIW3B7pDoKdOcsCxDBxQlIWPKrHGjzpsKPCqGmBAAAARwGeW3RH/wAAJ8A594C8tObDV2tZPGjjPZg0barYGaTc6ayJ9teDVMAG0Uoqr0C4gyVIyit4wtNTIbl+mi88O8VDjU3Ey+yxAAAAPgGeXWpH/wAAJ0DyTWEouNOd1ScVhZ2qYG8uX7fQKoPlbXe83SoFWmCufQXAxYuW3WggAKwG8iqOcp8ad2BAAAAAaUGaQkmoQWyZTAhH//3hAAAElLWKbH7LVIAo3M1s46Fv+PW1BFkRVv2Vox6sEqxdAzBCJvt2qAFj35iFyyBHIe//va5y8CyjbahjPkUCkEuGs6FBRA38GeZKGrM//1ypXt+zOoklRlW4WgAAAEhBnmBFFSwj/wAAGR/fE3cwwQAObUjrr6KHdRXYlx3CrQJ28ozwQDWP778o1NHMz6UWsnA4w0G+G13SSqg95nPU8xT9a7hU48EAAABRAZ6fdEf/AAAnwDn3bWwvB7JPjvaFdiDcWkqQh/wDsMGyKzu160oMxlsg6qxyZQkN5mogAHvXaNguYM9KaYBynD7D+dwa1bN47S7Hp15sbWPAAAAASwGegWpH/wAAJ8UdzuziVF3U+EFtIla8htV3OqxCBKOpIfLJIKTR239noKdR+ioAHvImAYELjmIQbWRQeZRAC6+aVL83kFCZEQdY8QAAAJNBmoVJqEFsmUwIR//94QAABJuiamZhVQCaMWAIYXWmb6M+b0QHMdGYzyAxbLmrckkYORN11hAwLtL3xtbNDZo3BkR/2xYSHTdI5QQPPsTKe4HAOdpOJdA53ZTELdZNils+BInhMGFjq2m/EB8MlXoJCCFnzHimzdJJ3spWO9cIXDahqC8mlKOUwn2uPu0yL8OF7r4AAABBQZ6jRRUsI/8AABkfPS3sfrxmg7ozkua8G9l+HfiXPdVzc46MA+lZY8SaQDJdZzWd+mEDj52ooRK6tHTs9A3uDcEAAABJAZ7Eakf/AAAnxR71Cs/r/GFwYxBiXMmtdIaOQqycjFS8NjDjhTG+dwSTakfuyoOBQOOI2qy8b+VViASge2yHX59+LzTT4vWbQQAAAGVBmshJqEFsmUwIR//94QAABJuiamTGrQCCtFknT6WwILkFH4MJ9Rr1P5nGviKoPgzki005bql0VtSZzrvywN+6VM88iIAPidcttE4g7V59GBdHcMK63iGAAOgvaQGDjnivpxVlGQAAAEFBnuZFFSwj/wAAGSJOoEIrS/j1mbh4CEyzawKmn40U27sTNbZeALqr30OuXa96mh+bEEXFozvM3vKJklacw06m0QAAAEYBnwdqR/8AACa9lg0ibGiZUhGjDPBPPgUttCcIvEnuAsYDaboQAVHLRWgnTA5XD10AEPua8vGdmKU2LQ4RXdQQzF+qNYacAAAAd0GbDEmoQWyZTAhH//3hAAAEdRS54eEX3wFQ5QyQTQfPjC/cuSDfYHh2UCaFnRvesewjd5ftUphUsvP5UA3LiZ9gwR0+iml1Tnk6xCSd55Br2zknyq2eO7u84rYFnWVdb1PeRh23JW87mkOyKDeJl0SBRVRXIsABAAAAQkGfKkUVLCP/AAAYfz2oHC07hP9kwAC2iPMuNWZvJZ0TMIiVWggRdj/pkSsrXyLGNlHzkvVJrvIAm36sIR8Mk9iy4QAAADYBn0l0R/8AACapP3YqTwf8URpLYF4wRUk7VVrONvMtQqbpge/PYsdL8OpYAmoBKnUUOcWDxiwAAAA3AZ9Lakf/AAAmPcr7o37XL2cP3/qn4oHFZZnfARr6BJBqh9VNgsY7kJXzonnPjbtuzk0i0m146wAAAF9Bm1BJqEFsmUwIR//94QAABHuiamggFwjw1GkAALh5mbZMAN7AfRS2iolbM/cq6+EQRYSOClHCxWSd2VjULjzgMGFkpDrvRDPihRUD8pajRdtyRoMtYGNnmXgRf1G/QQAAAEVBn25FFSwj/wAAGIhmhN4A5X611YHJLyoJbm9eKKrREkxDmqGImr06zp3aw8ikDLJpWvKhr5EFlafGWsCRU3vzFLw9E5UAAABBAZ+NdEf/AAAmqUA+9SVrL+gx5IvvbVMzBRn3ilgSE+Qa5C2mcl2NSHFxCACWLfqepTBpyMWpL1VYRakQvY2V8qEAAABGAZ+Pakf/AAAmrUXz34HckI5NrUm9742ys4TinEPFVCVrGE1dp51Sa/xOaOaHcTgsyIDAAnbdbjno+4lT+xh6nZyUiojZ8AAAAJdBm5RJqEFsmUwIT//98QAAAwLFD++zXqUv7swIh9Vw80Br4TqBb5Jmow/Kqd017C/3PENWrL63poSyAKnq4E6xXLYiYQTdBzkaaAp4Mb16cBM8Ez0L1lDmCoE7X6cDDI5oOaPjSbNQmSqIEWl5NX6J7AxZhQkNGzWREJ9pSm5GAtaVJgEIfC/X7TqKfm4MewkJw/l56XjAAAAATUGfskUVLCP/AAAX7XzNsCcTDnHN8/CxUQjSaNLJ4gCXHqZ+9nx/egQU0hTZndkGCYAP51JNXPfn783N1AU80xhvsO8PW+w3+R+zl7hBAAAANQGf0XRH/wAAJalA1KiMC+52tVA5SjBrUhi/pzJadUrlmcfmxNPpLQL0LzM7dBKFoiSClsqAAAAAPgGf02pH/wAAJa4+QoAOM2w+npqusCDAzuukOACvw68YzLH/KgZMBO0geXSsQl99pNK8FUpu1nzgjaJIQs+AAAAAcUGb2EmoQWyZTAhH//3hAAAEU1Kjf3T7ygIAiaE7u+rrFspvvG8B51ZUddEKtDkBNECj0qRTferFPRO0yJSbVHkIDtHwK8gkLUyIP/3WjN9hA1o9mdvZSZ/JX/fba9JlhmcddiumF3eVck8qtYTdmxGBAAAAREGf9kUVLCP/AAAX328dEZTIHgKBjxK7a/moOIRFdOIYY3VAQRiqMA/yCggBK+3k8vWHRptQ3/iU2SFb9+WFQgwEX9AYAAAAMgGeFXRH/wAAJak/VruxgEXkPSSU1vN6jCfKVmAxLs4OCUSfvr9oa70bhV36NL7WfOBJAAAANQGeF2pH/wAAJa1Ef91P4o9IU2hPzST1zRclHSWhsMGVxk7NZyvSizifMMO3EHNSdBFjudnxAAAAdEGaG0moQWyZTAhP//3xAAADAsfJ9ZdDyFgC/2fyE7Bleal7U4997AG9EJayFJU6DwKP791FptR3T/55bAKQ2LSzehjwNzv4zeJLPivCm+rGbi2L7N7NJEwkmVcCqoonkugSfbBxGZ80MaM6USaqh/rUNvpnAAAAN0GeOUUVLCP/AAAX4nc/XmDMHdXQNgE3FSB26RzDuSAYgn8yAgtxAzQc6A1rJy0ei9A4+Ib99SEAAAA4AZ5aakf/AAAkvTAGfoweAAX8VsWgMZZteHPYcpg/ZVZsk5UL8oFtpR6JueUnC8DWpKabZnjgGK8AAABbQZpfSahBbJlMCEf//eEAAAQzUqTrSEAPiUzt3Fz99M/P3yTWaJUTyvm/2uNbRxBgZpIcuEou5edJ/3hF3AAEUEG6yRie3J/oD4Dp21Cs2RvX/qY5+C8Uz9q7qQAAAD9Bnn1FFSwj/wAAF0I0BRKADjdSb7AH0VfD4ntsZ0fxc97Rt900KIRCtE0/sPLTtFhmg+cU+zSlU7Y85C+FhYEAAAAiAZ6cdEf/AAAkt6suPco5o9TtIvPhUlf9QtAIsr08fIlWpAAAACgBnp5qR/8AACSjXSUXU8dOSJnLA3HtLoNBM0sD6MAhTfJLJM8wxteAAAAAWkGagkmoQWyZTAhP//3xAAADArXJ9lZuSId2W9AIeElWyyy1MyN4V6f3GDe15enuDiWlnCYPy2or7tyfbJrBQiEUqZtsEeQHmz7O2Nil5g2GARXwBdK/l8ujgQAAACVBnqBFFSwj/wAAF0tfVzrZzo+M6lN37SC7yghXwiMQgifPPrXgAAAAOAGewWpH/wAAJLG+YrOtlXgYGmUesqKAEfnIAi7FspyJwEoaQ9/MuR5iYUaAFrTtJsv0bjbUy7rxAAAAlUGaxkmoQWyZTAhH//3hAAAEE1E9+kgFma6nqAksWz6O92kmO7LIoX2YWXxcZlFAxBXhzmnvJMyoNtFNPCumgVkAzsgYtxIHOnQ+AnW8cGaJR4YCPPsh3NxEC1WtZC+ku6AsFCkl3mXV7h5wen7sqi9UqlVn7arUhhrGAPh52x9AgijNgeMKM0rArSsMp8x+MNAVeQQOAAAAP0Ge5EUVLCP/AAAWsjA74ArfeQss26GMt3+TobYGBhnBSWutLW69hTikXhk/1MhqYD9+kN/yNDTfSHwcIqCDZwAAAC0BnwN0R/8AACOpz2dp9QdR0yWAw2O/pLKYfKCnpb7eKIJP2QpBN40nj0zKJUEAAABAAZ8Fakf/AAAjOz2SqfwRCiVsOHaX5W9aPAs3E0P6ncALH+3vqaqT9eR84w9+GdS0QhqSlwFHq5zDJNiPpssVswAAAIFBmwlJqEFsmUwIT//98QAAAwKjyfWXekI4duWF6/8gBsVjN3a/I9DTGEs/YnJ1FF6hUX48XQheB2Twt7bolghxkxxlfReCB+M4Q5xuz7gpQlYCvRyoHzBgSdzTjK/map0zHqSlXGiXiSSQGvZvxJFTjxT98E1Do67/eIsMRi8OJEsAAABeQZ8nRRUsI/8AABayLwj6uwRykC0Zdq2oIfSEYSf8IAvKcVwQmxp08SGXewKselEJ2bUIrRpu72vW4VypWu/cMeIHTm7NMZVQbA0XIr48Q8HATAZa/WVmB7cgqIGDegAAAEABn0hqR/8AACOtRkK9G1mS2AEW+tESo3110NkcnIJOKV10WOeBEIGj4TBTqAD30ydDUX234KF2w5MO/oDhsRb0AAAAS0GbTUmoQWyZTAhH//3hAAADA/VFWf9cLLSLrX4H67k/cRxznsSYc23g3l3/hD6slye7cfk4kOaHXWfW8PQ97cxpb5cOj/ho+soasQAAADlBn2tFFSwj/wAAFiIvH+iP1O+Y1oC/5aGMOgatviZSaELWDsAlZT7oSXdcuyFGCFt9geGSSLXO7egAAABFAZ+KdEf/AAAiqUC/6AVT8imUiLcwq9QA/Pu4cDPOJqfqOBUAypRmckPrFMWlbylx/5QAf3UXUJfgaOWDKk8VBHqY9ft6AAAAOgGfjGpH/wAAIq1GlswJioHQKaFKTuRU844cLUAWFUQALnPN8jH5kl6hc+qPwvrtMyfJQxKSRmJa4IEAAAByQZuPSahBbJlMFEwn//3xAAADAoztpNbRB48aVKfdMKEPZrGJuXLKEsGgABwjTkyu4IGHfEHZG3J5Q45Lgu7cBiUP0nOOcliklWkartfzD4ZpW9rZVIKxlHP/dbAxyQECv6Z/yo6DxHkDCkaOcffhmRKPAAAAMAGfrmpH/wAAIq1F5nDsMze1veUzr9mhs2p6ENQ1n/D1onQXHKCCRmMPj/DpOci7wQAAAIxBm7NJ4QpSZTAhH/3hAAADA/u/6mgiVUA1KMZx1YDGm9Jt8OFbT/B4MwWnoYWu3HoXh2Zwjcz85G4KGxoBesIdr/ODX3lyj0fV63WephAvCvd+f1MnkUVy+38CkxkHcUgenVvAtz3ACExpfEatxC5Uk+DYaHDwA5g9oV99UW8Bt0UzBIujiQYrcGW6EgAAAFFBn9FFNEwj/wAAFislRTuSaIaLHwHDgfJ4BA1B/VCo2Pnq98g1Dr3wAh8Ln+B46wk2I5rKxtK/rEI7briIk91IAIVqkTF5p922nhHTiSaNcqAAAAA1AZ/wdEf/AAAiqc9Ou1e9Eo57pH+ZZ18u8Hw729dhQPi92Y7nonsKK/AVIKgkK33Cj4+MZIEAAAA1AZ/yakf/AAAiseddra/YQT7VJz2ylgSh+4Y4taw6IsbAIOqzJxZMWunQ1lsEopa8qoxVLkgAAABZQZv2SahBaJlMCE///fEAAAMCfKethMJOpqdDH6+wsgFWUAFDpa/ldHDyLiTh5uofqeR2VLm8ReEKWTsvAAVSo+eBmj4/B2flz7m0Q256OkdeMs+rdMyHJzAAAAA+QZ4URREsI/8AABWHrp6th4+iljO+cNZzm7Sbjv6+o7BNt5V61XOIfhyPndydPMNwDPNLexlf9mWCz8jwXJEAAABMAZ41akf/AAAhvTAGgRvV/S96JqZgDy/Dlaobq1vrlIaCQpQPXJnnekiriFthKWkF8NC1xfjABut1uPE9PRmT/9TRIQqGNflHDaWfkwAAAG5BmjlJqEFsmUwI//yEAAAPI6OuoPusX44Z41YBDpZjKDK18Bct1Kw0Y/uFH7nPMN97lhRXgMCvxDNFTYnf1AVAAE7dCtmVZPsv5EVMYsj0bnXxd0OUB6l3GLCORSx8aWvIWbTPTv8QChswXwopgQAAAF9BnldFFSx/AAAhq/0f40YBlpIhqkCGqnsikNZ3tTC/MptSUX+naRtsomwx09GL4unByzX/DfaInUVmUJ2XYsAJX0K/B90L3Ff95xdWdCzASD9kq4NFS33dC3X82G+ZnwAAAEkBnnhqR/8AACGuMcxPqg2kNmrZakRcX0Vn3qxs19uJkjYwQs9R68M2W6iMBPyeEnG+MhWFV4AAmwKo9lw/MHsvnTNCvzhfsDuAAAACs2WIggAL//72rvzLK0cLlS4dWXuzUfLoSXL9iDB9aAAAFCAAAAMACmzf2NA7/EIo+AAABYwA5AeIYoYAiYqxUCUcq+BzuABdLmw/8j+FlmyAegdvwQrOSSzpU7NmYWNiF72pdi9Z3LvfENiWOHKgY9A4H/S/W8VbeIQ99Nc+4uRsehN/FN5OLGQl9xMS+dk/jAxpN7aKru8yZuhmyhTKvlmq4WL/yq+3or81r9mR2hhQZhbdrZZ3Ga2XIW2zna57VYSRhaVbEWKGLhZqbv557w7yNIpG2SmnO4acOhTkFJ+F1oZVfEufAgngehsn0h5AeB7Rxky8dzuEl+MPn9TsclHQHTzxLDqME8VzoIVhu5Jb/Pr6dmr4k+3r3i5hIFrjmmHNgof7YmgARh9IASLMyShNIXNa9/PkCscPf7dnSiSA04dUNA3Ghuh5XZ6xnEnmp4LjOPu8x8SFKWJpIWRBL3SZi7Fn6VE6dEiHqwq+I79yUiw2jJaa6V+2vaus1jykbpXWklewz1V14DIJXP6X2ZmgKb/K9vh2lwxWVh31E5Y/PbsHROLkL2mA60z+8oC4D3hXK8IArbX6pirubMeOBrqtXGPICgRz8E5LEIYHRQPaDcc5u1som9oy312PWsQeQ16kQkPuv8AjNAA7/gE7L8e/orKUXK44Eg2fd/ebaZWR6+DlmkAN0v26dOqeLKi870oKwwdzfFMnmcIGG99nYG6CfVTfiUfQTwrdLcZvlnL22vqYy3gSObalxA6YhY6d2gxN9LtACE68+0ys2tluf1Hwav/OlSxmshlMpFfbIi4tGEHPHobkO9jtwtgA3yDigNAZ+mnFZ+WaCBx8GdnHLIEBEyY5zZtqCCgtxEZ9d8/otp0ir/R22zDrr5L6nRUG7TIxzcDDdY6AJzwAAAMAAAMAQEEAAABrQZokbEI//eEAAAMD5b/qiTJWgFqpnYgTdvqPd4vZ07yjPeSh1IsL6MmEtE6kvO1m7cBX2iSskWO2kgKA+XCox3BwGwqsSTCDXe+9NfzyOXWNbhPCD3tGqM3kUv+juNy/Qe7sJFWPv4+Is4oAAABKQZ5CeIR/AAAVlSTirVGASufJ8CBn3rsQh6IBPypNyrHGb/KF8OcFHAhNNzf77XjrYzOQywgDubERPO+oelBxoiHlbee76KTM3TEAAABUAZ5hdEf/AAAhwNGt0nGlDIAa76UOKBn8C1D3/1AGI3oFndgSy6xftFo28lDUlSxk0xrIoox7lLz+JxYwZ6Vf/qGKr2U45Eah5AVLce72EqwVuemBAAAASwGeY2pH/wAAIK4y+dtB/jZij9R54x7kyJSv4pMSR7MVFlQRprrOmsHGNYAQGEdx44f+wuxRBObiFQFhgKfB9zQfUMQQ9x2AdaLdMAAAAJhBmmZJqEFomUwU8I/94QAAAwPJNyeEAUfFb7SV93fOuT8xAdpcNiQHTWkxOFoMUk061iiBO17b9tMePk9oHGcKJai04Pbn5wMRfYaQC+4ZbOvGRIRulvKO1ECtBuLWnRbWbqYC9/25E5pw0di47yLU0KjdLzJxFuu4iUh5npglG3cE9Q6QVLNB+IuK512vIdcJKSGRjF/x1gAAAE8BnoVqR/8AACCt6m6EaQg9DtCO2o/tDvOHCjAyv8CUyyeplgJtOLEWC+gqADw7gAbUS3sjCOcj9YQff4XupvToHIu1KSWJ1XrzqUBGaG6ZAAAAm0GaiknhClJlMCE//fEAAAMCalsRUSvXV8VlxMIhojdkauwA4ngBRjpkTFv9qvVxVZyCI83ZBfQuMfUArRI1loHxywZ3S//VvewyOjJk98YBrhFOsaFdM+duiUovf1sMNTl5suYOBKoj6YuX1MX6E9/wW4Ge6LdlQgS3vIldCZ5BhU3SsdOvll16CPPwvTDQQTd1tbb64V0VIXzQAAAAS0GeqEU0TCP/AAAU+AbHsIZ0yJRbSUUe6VhmzS5BpaKszVWHPf9cSGOCl4f+GLsPzZlsl0K5ok6I0N5v68ZwTU/ra1Zb5Ask0GC9GQAAAFsBnsd0R/8AACA9a9AgA4sGpVt37wq4E9/M8SnhjsOvO7ZoLOWw0+QMrN9JQHBAT7u9WQwcCU5hN9SkLeY+9DcgzToELvQ/+MDmBQfTvaSdjtpqnfHiukaTgV3QAAAATQGeyWpH/wAAIL2Vrbpq/YXlV8i0ubv4kum1LNM7EGJbp/Ne5nRvrwKr9v2bUVbcEyABpxOAHN0YMibYszn6rHIwoCv6iWOlwXgoo73dAAAAiEGazkmoQWiZTAhP//3xAAADAmm72wmvY+AfOKSvE65x9MivUwx8rksx0T1e0Bb0jkAVguaEAPfeHn2Jt82FSWNDQ16FH5VwocEUlj5fbkHsMY+WyTbLJ6YNOPGEuShXaSUXXzPdhp9SBeL+YD127RPTYLg9R5BWVTtl/KXSEKL4t5hulbfLq8EAAABLQZ7sRREsI/8AABUDAs8AI8CfyzqK8574a4Eld96Q9qnO5btN2P3B9zRG5zxxv2q2xz0M9tgvk9cpx8y/85YfKGDsLoqy8G0EJX9HAAAATAGfC3RH/wAAIMC4hPvyNZfw/Ts9pguESdAP9itEpz8SuIoi+IUZaUHKybEjmjxq9gU3obCqNgAlU9595QbssHnxXQl/TVne5nGZ7YEAAAA7AZ8Nakf/AAAgvZPKZT+WlfDPShTJ9DIwW75eclRbVqkah3wX6D6UA8h1pj5ehd7oCLMe3rxaTsdh7oAAAACsQZsSSahBbJlMCE///fEAAAMCbdEh6j3ySy2wDUEDMOV7BhlkBUUhcLm/4Dqhdjscbi64BdfeK2ZUHN3FrlWyEjaKAqiXpZ6fIUTRQl3kKYvm0HIPV+gY/UKEVtgob3Rn0UrvTdig6R8yqTHCB2sgHVLCCkx+49LEHGrihdBaVKMwvqpVEjkestlyrlYGlloDLk/7A9SN07k+mENQ8f7d4s8IzYCXzvSKe+7BgAAAAEdBnzBFFSwj/wAAFQuUEoyq7NVauumL3XhWebj/f0mT24TzAZJlUpGedzALBZ7GKX4UQ9pM6gkYO4Xt5qbwLDKETdebrZDUWQAAAEABn090R/8AACCsFQ4h0japPdBTrVs5o82jQOKbrvMgXWks0Y3MvANEjf/XSNIl/TduLXxPDSZOG9fIhR/otu6AAAAAXAGfUWpH/wAAIMI96MB0qNBeP15vwmt6eyAH53wIhLArfNgx3KEqTnyOPUIA5q2JqIcG9BLYPk+8sAEpSmpE0baMkLTDToWH8n1AbGFP3xHLpPvcyY3sPaqCYlJoAAAAk0GbVkmoQWyZTAhP//3xAAADAlwQwYqf34ANvy1akVtsKwMxXXtbkWEf5e/tvRXOEx1btykA4vLe+w2vbKd1ajHwf6VNNtiufIezDz/+4xxh0eEbnOUw+NvmUH/Lt7pg7w++E3q9ORvsR3hNCtpq/cyRp+ZfZDr7F+f+jmYLOFDpsI2kJdndiFUlQmUpi0daJ2Sp4QAAAENBn3RFFSwj/wAAFE14HsBl8VwDI07VffU/6DkPG/zHQB4Xr6vznDb4O/+Q/+w3X5cQc13/tqQ63B96/R6DoWv7T99xAAAAPAGfk3RH/wAADDltpypSSWjDQcreSWN9NyOfeHY1A9JCNi1YLRJ8cAAPnPUO8iufbv87E836W7hXwfAcoQAAAEABn5VqR/8AAAwVYpPVqZtCFKyYIHIBhXz5ldGNcZ6D35pzehkrkH6dGZobXxDOmxQATVngUcJThisHw1PO1V4gAAAAmkGbmkmoQWyZTAhP//3xAAADAlmhjABt1soLvKTovh8Xzyi5DCf8JQNQbugj8C9iNGu129j0vB+UJslw/N9+w9O9eWtBr2X79v4e96ZIJu77/UwXyS2GM3FpDVDiP5SK8Miasu9beG42fm7R5IsqDyliLT/iLYMPQyy2PrVrRmd/JtZlyX5P6KI2/YgRQEO+EdPdWK+Byrz8lNAAAABMQZ+4RRUsI/8AABR8nXgCIWpoAjtDbvrBArEtmSd+y97296R45WlOd1z1p3g1w8pm2LhkpXoFv60zyCjU1+djObe+JFnsavQIBEd6QQAAADABn9d0R/8AAAw+EDn/fw/Dv5WTslowKqMUXk49YoMeXY2s6QAmrdJ3QEhmhq5tItoAAAA3AZ/Zakf/AAAfvNUmYGMM9VDhXawUoAMJBOcByHl8q4B4u0erudKoEYVZZJd7o5YGgJ6kk+iRwAAAAGhBm95JqEFsmUwIR//94QAAAwPPv77pZ3fgao3wqS3ruVZYbf4SaLm9qHz+eSfRoKaUx4ybu9/jFcv3EkZqENXs8HPyC6ZYvAIGky/SMXC6wPhLkLuEn1IMdsmg0CaMyBJ586CFSwGlSQAAADFBn/xFFSwj/wAAB5VhItb7KIAJgmQGWaI1kanUovkSostmNpqpZkSr8O/YhPRl+wP/AAAAIwGeG3RH/wAAH8sbiYxQQqX5ENGHUsRHJPe5F3oQdbdpoPLBAAAAIgGeHWpH/wAADDlubIbWm+C08u4r6hF131ov6VNreABd0y8AAACpQZoCSahBbJlMCEf//eEAAAMDyTe8S9KeFEgFGeJHFwzrfqz8Kxv0gCeoLPQHjOxKTh+a7ozOad9LA5AfJtGEQbz+nK6ZQRY3a4y0isJ38JW92Ad6Zao11GSKWMdtR9lcSV3TeQvdjifdEnAR2jihHVS7070GcPPTiuUeFBOvXCD4CFKusVxf2oaBkedGa8Pfn8u0B4huPMDSmMD/rwYMIa5H0fbN8lgcWAAAAEhBniBFFSwj/wAAFQNHRd/2JmSM+6HPHougAa+pQ6B4BYs+zKssdvARW232nKD74ABx1VM0lcVd5Z6odT39wXiDWOgAapHqRYEAAAA3AZ5fdEf/AAAgwsxyQFYInn3iRvRH7Us1rSRhGZpkK0/VM3r7ShOW7WjZy/FOMhCpUO9eapwxwAAAAEcBnkFqR/8AACCx6DR5PpvOevxHqgkLALBXYDiGVEbKpuV2MajzMn5/21fvyVcjN4AP014fM1vz5QsZMWvtmZDv7GZqBHt44QAAAKZBmkZJqEFsmUwIR//94QAAAwPKgwz1KUAovmvytmA4rs1S3I5BqEZbyAwSMfqE0Ccc0R7g8nP2zIe1wMz+ro0nXN0NLhkyXnXf0EGG+uK8WBWFf//AORUe2LyoBtI88RB2QeqlFWTy90uRNn85mWHecHdGLILMBzZfNiYMlpOb1jfg2DQstwFivPevsr6/yH+STMKu0+XtTkaDsleP316Umn+if8AXAAAAa0GeZEUVLCP/AAAU+q4oMfTf0kgBGRpEl8dCNA1ac4IjHxYv1O+TqyUU6xOoSdWXSAGyjrs0pys70d5MPmGb+81V4UFjLHHxslxxQ9qA8Z+WUy5TkW/Ho6yaHd+H3Yp/PCZJQ0CdwFz9d+u2AAAAPQGeg3RH/wAAIKrWt7gM03nndTF53HBBN4xCTmNxk/CZV3Rkb+5grfCIuj0NSgAbOr2mFsWsO4o6/OntHtgAAAA7AZ6Fakf/AAAgvUy60eKOhCxq8nFa93COt1z2JfF8lz/wStOjc2bwO0eppo4bNQ1ydo8fuhsJM2J13bEAAABtQZqKSahBbJlMCEf//eEAAAMDyoNVdUpQBtGev9cG/cww3bOiZDo+q4HyuaaH6P/xEfF/mnx+uzMgAA9kKU74wgSOgTW7SY8LBL23mp8jWo/mr17QCXMi0RL3NfLtv4Cb92Wu7pp7+SHiA/tpzAAAAFtBnqhFFSwj/wAAFPTu1nL2kS6bgRlGGZkz0NCPtioN1tcKAPbqpW34J8ba1YTe2SGO1Ic+O6ZolxWm1wAJUVAcq2XyNnw6iYwK2hIH6vLt7vJsONnpYMLye3oxAAAANwGex3RH/wAAILehZTokPiYDa1MPq9qzvgl6gfVZfEELrlV1uIXm1PQd4A5ufkMt1nD/xuRSbugAAAAyAZ7Jakf/AAAgO345y1efMFXL9WHkugVx5Sgwy2z6GXgT8jb9SxXI775Mlgn8VPLm3dEAAACXQZrOSahBbJlMCEf//eEAAAMD30NZlECoy52Rxo01phvhzrmOEmmRu650evN/5fMvOR4NOo4DjhUh1CGQGH2vhXZxdJ+EgpHjIto9xuKzK7vo6shDC4qfyJI4Juip82VNLjbV1ryZ/H5G1qReC7frWSW4zWlZrW5HVBD+dXpkFEbvUOMiTBM+psGvdh4gyme1XF9hdv8zJwAAAFRBnuxFFSwj/wAAFZwUmYpezB4OyK+lPWy9pwHVLZsVAQzvQUSw6b+CQZktx0hj4juDdkXJGH7/2Wq0bnPIth96cO/Npysfw56hTD6uYeefQW28VisAAABDAZ8LdEf/AAAgt6dSRpjh0gZg509yiPgqSlwARY+cVR+PqfkQuPN2kDEALM6EbNi55icq/Zedw4Qg+sVvNdW5aePlmQAAADYBnw1qR/8AACG+4jb4Wl2nvbclW6vK3cEjF3r9nB06kEVG87rsVmNMVHRknboYzxOw0gIzDugAAAB0QZsSSahBbJlMCEf//eEAAAMD4EiGodV5SclociYzjkAe8+Pszed0O3v/LDUMTxA9IibX0d8V4n07tuyX7zg8nYyxiSCQVNDLs01UzgLparqzzD/F/VaTkzWdb2umtneRc4PveNrSbhVIGuKyAB5rvWyQBGAAAABYQZ8wRRUsI/8AABWSHbfgDjSRYhO0fS7ZRra1LXgYdX67CDvxk1Qh7Ktjq1vEh9GfA3hH6t8JRDzJkBMz/AgS8Q5pduAbhk/bzSNukZ67y+lG487nT5F/NwAAADYBn090R/8AACG3quZdb6cDXu5ODJhx31nd0QFF4FkuNRwmVRIbvyvDAZt7TU5nuaA/CWYG6YAAAABFAZ9Rakf/AAAhnkSf8P6+d7NJjjLE5MWCOsT6vzD/LBnM4S+ZsupGS9taQKJrA05DmfoAQod5x6vrmsgJvQMA5Ud1uW6YAAAApEGbVkmoQWyZTAhH//3hAAADA/VF7tMQGZsEBWTsHVtfijOoAFxs/xQd/uw50ssTj5GusIQid9ndCvz+gwg7nKmHWA//DiCkFMfC2dG3XXjWqr5MBfm0oOv0FUg13MUYlw9c1+qtXpebwehvBYiA0gP0xb4W07foog9Y6oMAE39qp5xQpi0A0GxrLh1pzBzzLE7b+o4I86W5e7lneHJRuysh9rlzAAAAUEGfdEUVLCP/AAAWLEWL++LVWL7CZ475o+omHO0yRQ1Ff0JyD9LKn7W43xKSNANjnmR4AP1tYO1kdYdIJGfugUT4J0+5WHlb0OV6XT4htHTBAAAARQGfk3RH/wAAIahzx8AIkt/29KtQ7NO8SV3PKM5NS6DGailZwbpPA8fC0d7T/xsltJ7NkNKhk7+O6SEDtA7rg+Ka3vGVYQAAAEEBn5VqR/8AACK+9HEM9SjBsMbifIR8GJARsgn0wVcH2fM6g323M2dGPJ0RXkGEAZHInwFECtJxoZXQw1LTr4QKOwAAALRBm5pJqEFsmUwIR//94QAAAwP1RdG3wByWcH4TRtP3QZbMIWC6CB78DJS9/1Jy6yr3xnWTaeSkCzOlnDx23S7PGUfobwUJKI+/hgswjySN+CpMbNvgRadUIj4gUx63EbCcfgs9yict8UepT3XI7Dh3VAudbhuBWvJHTqPumQFclE7c9c1glYVXl50613LZhW/VpUPktuligjQxNyfsEo9b6CD7Beeniz2oHKB3x7xZq5KSvUAAAABnQZ+4RRUsI/8AABYiQ6eiAI5tmlmfcXh/I0kPcuqTMuiLQBUy/vD01LzJ2pp+jMQ3dPdUTl91UaNSYSSJ9wHdkIeDnT/dxvkbgMeXAaefLfFNxXIdv46lsFY+DcXVofFZcQL8zEGUgQAAAEIBn9d0R/8AACK/7yTkLZpViKZUGyoS7qBEQDSw7VIppQKEV7JWsVSPXm0WZzIRsV4PjEfpq0qq1Obyt7EVzXeH8mAAAABOAZ/Zakf/AAAiOquPtlPT0P3ktDnDF4GT4pPnC25GpMw3gDZmHBkmIUrXSY8qJWSFkN0oC7V1ALLspaNJNdv4MAK1DgwpMPiTPwvJ9P7QAAAAhEGb3UmoQWyZTAhH//3hAAAEE1IL9B7rRa0tbuKaTVmww9nIFJXdN4RH6gBuOe6RTNHFHqx0vs/A2ZfBBHa9I34du2z4fyusizscwmkDTnohHIqCpm7ESIoakbnPAlEPwNzrVfTBHGcsNJMJTGJQdsrOKSGtPUA+vusNE0uPZQSgjjPc4wAAAE5Bn/tFFSwj/wAAFrw8gzWEB/AHscDr0g2XfcoUIA7FLbTX59pKjiCSR7G01BPlKo4fgbcJURm2rSrjZsqu6KVVqXQw1Hx4bdWfK/qnCYAAAABHAZ4cakf/AAAjvvN/eqC5ifQf+qR4BKZNDw5zv4iAFmU4f/ky/OIxFXQLLfBT6sF4bmSufWz1GuP/o99RuPIH059qmgJg7gkAAACFQZoBSahBbJlMCEf//eEAAAQWQ1L/r6NgZcdZRI6pefwlDWxEBRYOztT7N6MwGHj+h/4c84XPe2EIAX0J+vUbdRNIRUO2vPcSBBEvgpCGsa4xJ8XAwohdo6wE7F/wRk1j5CvVdtn95W6usaYtKkwy5OFMy4hH6tqPfsSIAKln3k6EvlHXVwAAAEZBnj9FFSwj/wAAFqTs18KKxOLBYOsCzOQQJ7yyCGG1CwKVeBBtYjATWh25JshWwUPs8urwR3t1gAEqwdoelT8B4kDjyGCwAAAAVQGeXnRH/wAAI6iOi6hGeOzUPYzzlcbX3g5ym0fI2AkVI1iikACP+DLw/fH1jBdAqQHeUx5PafvEOoMA00cP9PjK/09Tnzcgrt/5H4UapiNSydoPebEAAABHAZ5Aakf/AAAjvLL+163ZrESMT3wLBTbqHnqMDyGo+hqnf9It570tpd3JOVOZblellwAONutx4np6NOx9V4RZK7850JCS2zAAAACEQZpFSahBbJlMCEf//eEAAAQ0uFtmKADjPT0vVVYmyMKXwT4rAhC5g8n7TD6CU3zQI0XTcBCx73/oGoRBVUuLigb69AkQUTehmOiCu6tbKIzymgx2799rBCBFZuiJQxNhWNSBjHLwBgCUwUF55oXHtYTqnXWNigmAdkuh9oHTuPZR58JnAAAAW0GeY0UVLCP/AAAXQl5qSJ9EACw4xHpTJ/2EvbC9S1Fv34GBaSkDfjak/wriWY+WGq3pn5wE4pTi/FT2+HKsULzy8QvimeK6dszE33qi6iLEkgSJO43S6jO/MqEAAABJAZ6CdEf/AAAkqUznmBw1oq9+gBJePs3PqW4Gf29UeaIsF58+X81IrYv8d+kfeNg60ph8NWN8fTAJ5optJ0nf/R0d7kg7KoaW3AAAAEkBnoRqR/8AACS8swnZnNZusjrgGif7XBtoh8XXpkw6QKXRVjG3ZypqSi70DbEPqACY9BvoFifFZr0w34Tza+xTpixHR5qcftLYAAAAg0GaiUmoQWyZTAhH//3hAAAEVLhscteA+P+XtTZGZ3S4lGT8FRsLYWgq4C1VLDcMW0dt01UAVR7y1ZNvG12gH1WmLEdcu0/22qmej75olvn3HEhxdd0MEJF7tOPsSQIs3wpbgxP/iXDFbNcqihb2lcL04wPr7sP/+DMEBdV6qfZGrO+xAAAAZEGep0UVLCP/AAAX35iF+AoQpaeBhGt1FeSQAGO/myA4PzAyuGF0yfahGFFSAeoQwoGJos3tf9z0eB7LprwjdeItvRQu1Wu7Tae9EVlvcbyBOWIPqCEZp4/YOmK8SG5ca70K17gAAABAAZ7GdEf/AAAlwuQHdkzWCeFtpVclfrjVzHMzArVTJDnlFEy2xQE2t6ZYJy+ACH0by8PnLwiM5YKs7F77ZPrUgQAAAE8BnshqR/8AACWsCY30qlfSIMVamFmxVKyiWCFMGE2ZNY3HOCPSCOElMWkJ3cqbFdwwkRQoIwBvaWFAIABOCd/IurDhemR8K5pNf7DQZiLAAAAAj0GazUmoQWyZTAhH//3hAAAEdLhj0lmqgGrk9i8ZzHJ1ITPPBhh3530c98dUeVNTF7xH+vycX3WwOpN6sTzfIWt6LJU/csQk6UDfvga8oJsoVkbCSeg9n7XWaMIjlcdx/jY8NirfxGdF2mB6i9rF4HqI6eKhccY+EcE4ypSSlTi894cQR0EIRLVzOvDebqHZAAAAPkGe60UVLCP/AAAYiYLZzZoHDaW+bz60nE7s8T89UU2qYCU5RhJ0hPlufk+uBb92w8y/b+Tbf49MLjVkGWyLAAAARAGfCnRH/wAAJb8TiJ7fDk8eJTeKcSpSayQP/UTpJRlXq0VKOdmJfXI6WArGeG+bzo+VfscAEzDW6MxpZyS2nX/guTfdAAAARgGfDGpH/wAAJsb7H/InoEtDao060Fr3UmxskxyWtVA8yB1t0cvTtdlZnQ9cQ03CSp0peZautPABthr0C7JALjnQHZ2kZ8EAAACJQZsRSahBbJlMCEf//eEAAARyJamIps2cgCn4YYaN42gMsc7iF0La/+VPUnowsxSBeifw61T0AxjYjLu0tC9eMrz5++Z71rQrvm/BurbJPTHPChxJQ7ksF5XA1u76e4aY6qMM+sP2ATZixv6799kiL1yMhvKmxWWXID/abrZmvoiQ77qTdSLx+VwAAABLQZ8vRRUsI/8AABhzx9cxEgAIqn2p7Fym4NuUUUsJItqGlyFQd05ERINrVuP+mSKccxysMA8COb0KoTBdwIWpm2RsmkhaHPGqRFkkAAAASwGfTnRH/wAAJqiN2se7IT5RScgRMAPwtcANEoNUJdCbBp3GI/AvVzQUGC0dOAqH4PO1YAA4L1bRA3ylI3ch+pQ87Uz+lRECdcDe0QAAAEkBn1BqR/8AACa8sv7af+S4L9ez8u55pweQoFsgYUkYZ7S2eq5+zbtMieqzpyQ44NpQAJxeqAdrB1sAJJNW2HKt3Q/kAvOoZ2vAAAAAgkGbU0moQWyZTBRMf/yEAAAR5ILntGIA2+y63Gd8YBL+6eHSngcal+a26Z2NN8C+GSO4g6l8IdwVNkFXZIjpE3Ljoc8ImTdvjgrRFOvD8ZmnTZAhLynk5Qu4jJIGWX3gDwL1CaaIbWdzgw5MquQRCw0z0BGjvVth2+AGrMP9KQ1FX4AAAAAsAZ9yakf/AAAnwpzZXZeogd8EbGdOOex3jETICXBKYd/jY+Q4Fvh4R1IrA20AAAB5QZt0SeEKUmUwIR/94QAABJY43v5N0lQZnawAE4dNcNHwhQQJCQJMfmaGtLudGjlyAq+n/GrciCKa/Xd8ns54I7/Z7Lz2b0/w9a7iRDSUoJZxglvuNutI6jKQQsCh8tdne+YCjuRRG8ooahygIMJ/m7xfX4cDUPFcgQAAAIdBm5ZJ4Q6JlMFNEx///IQAABHLd0HaKinpYJyAfATu0wY+1pICrwb1Sanm4lM+EsBsl4+Ky+LE+DExpSNV94VRrxK5pk+LMKXx4l+P5m2S66MFVquTAIzTqVWElbqTvKmTq5lF42jltOjx+Jy+UWrJZHr1NwUfjj+Qz/kzY0AYnJSAL4iC3YEAAAA1AZ+1akf/AAAn4yVpPcOIGi/DkxQIKZabf68LPtow3YLzE96EkkTldKSC2PHM7JvbpPILdsAAAABzQZu3SeEPJlMCEf/94QAABLNCee8Ams3rMMoBN4zlvzr8pTyTJTxwWmdtZOp/4zLzLK+59jkJed3Bf03kbn2XpybpgWe+JOy9E3qDysOGDmCdhPB3epFrudBPMVZUoq9yG4Ekb8C1Co+I3JojzVFxRu686AAAAKhBm9tJ4Q8mUwIR//3hAAAEtLgk2wSABHsHB6PuaML+1tjnHIqgf5cZjSIZLMtIIs+zbIsz4DrLt8X9iqmhC7azVJ5pocGZNawY3rDRV0f1tGYe+65s84g2axQpxRWe2VgRYVCrFZU0ZsYG+jxMq0kXguHQ8fhIq8h6Hji47sCNQeKjCl0Pktp8PGmPxdT/UoA08H6XpqdzblJL+mmvrLUGLVTXUb9JpmEAAABhQZ/5RRE8I/8AABlzFWxpoZ2Y+fbrJM02qnZJpXBW9z82tGeai3q32r1uMvQbuZwHZdc3Kk/3aVcLuS2RgA1o9xg2U8l+Fbhe2kfvPDKFu8u47A63gVQSKiBipD4E+DCz4QAAAEgBnhh0R/8AACjaMStqKJt13J773oMLvtR4xvqBIwFcl+kk2tZRI0q59Rri3lry8j4tR/BiAblk+6IGVX3iFIRlOofUnH4XDAgAAABXAZ4aakf/AAAo9o6VqQ7znY1zc+AlnnguzyT05gLaL7KBkJu400yBei8/83bQPpsYtKtR1hgoAQFTo1a8/3S/UuGCxUtA+zmTW2svuDDMtFPy6G7lzPX5AAAAjEGaHUmoQWiZTBTwj/3hAAAE1RT9am14BqeIZh9tjj1gFn0t9enzvTqE4z02/7Xht3zfj2zCycudfHdHgAFg+bFjwixcWBRem67aJRsxXY6MdcoCAhQl4TQ1MqpAedoxGa2bTcfsruvEI5anf08Ef/mXKiz00mVsseD7PuxvWtdg2WDZd5aY9LinbdD0AAAAXwGePGpH/wAAKhaOalmM6NZ8jZZksrZeJdCWXEWp6MsIaw0jJhwgEykPjRN+8I+TuQ7yINU1SOkTYPgqPOVC822ACY9GCJvwhhDaxDlqg0v9we70Do2c3F6Ay1e4tcGrAAAAi0GaP0nhClJlMFLCP/3hAAAE1kAMdd0gC/2IRVxvb50MMSik1Ovsz5Kl19P2yqeHA4tITkaw3qzPTMHAyzRDiW6HBmyV/xRfaO2yLh4Bq2TQ2QZtJ/oi8f5Cpw9XKomgNrvMugABj+MxQaDfrYGXnkIRCtlHFf0o/WBArWI/oMwIS7zZMEUirCbqV9EAAABYAZ5eakf/AAAqApzZ6AwDlcxQn7UiwdPrRHzwSllpPhcDfyKUUSllr8qxZAjRh/Y56aqG0wSdxOnkMpNaTc0+JMJTCNv/noAJ26FPPh/Y9e8A3zTIManUwAAAAJVBmkJJ4Q6JlMCEf/3hAAAE9ZN3nvANXdwvqYtXUTWVCJu+R07OMjdAxohSaREt24vrL0bMqqni5SXGeuDUlBc7DDeRX18K0p9LaHCZMDZnbOQbUslEOrQCMdVW59y+rWN623/tSHeLmmELGtqbw8DXSlUSXDWzYf1/rzfQQWrzX44d/eNhSx5/EjJ1/YxqEbLZh3ovEQAAAExBnmBFFTwj/wAAGwl8hVJgud/1ycnhmLJQePbFnBZMTBcgKup7dlF4Ku6ExD7vFBLKNxqIwAmmUzom3YQE+ktPPyTA7FVWiJQRduCAAAAAPAGegWpH/wAAKzmOizo+VnpqeiUoV6juBuE6/M/t+24ChRvspziqYd65k8BKk6w2xt8y6EZLb/iRagS4vwAAAKdBmoZJqEFomUwIR//94QAABPWT+cgAJ/qdVG2yPiwk6fOBOrLR5VDXy31gg81p4/XnU1lANYVrmXTJ0Fn3j7lvQ90tAmU8x4iIDeVcvh1Cc7gz9zMlWVqxMUk8Qyd+1neWIsqllteDP/KJmRF+FfFaxpf6lVEeoYzMikzmsqZQqZ/sgaxz0GRNJ99ySOziQC7enlmLqeNpkYcgKktyne6wz7PHz7b5lwAAAGpBnqRFESwj/wAAG6lz4F+KCyW/A8s7LSn9L4YZQJSScgHo4W9Vb9nzzMid7vqtvi+fjFZKK5zXVQXrTkjI+W064NMnoUX9CZ3iid1h6ACHw70H4RX+waucsFp8uz4Vdund86v4gpacKB34AAAAQgGew3RH/wAAKx+e15X1lAi/C5wvcKISgACPy/4blzd5qtgS0Yctg2wWbIUKhDDZyuhW/hBq+AnUbJgi+4jPjrW6/AAAADoBnsVqR/8AACxZgNeHi0hBXYoLc2aVOSwcHuPVN4oCXzFfCrLXeBw7Lxb09hJnaBRHvAiHDDVZkW3BAAAAlkGaykmoQWyZTAhH//3hAAAFGzGSLagDad1yqTNXn+xVO5B+epHnUfpQ0lphKEpv16arqLqGf849XP1MWAYWOHODnu/YXy03mEN4Ksb0RH9rQqmUyS+F1jLD+cEm6CkRG5I+T7glH5B0hgIZicavpOfJdgWUO2PU04wOpiWWFSw3X0neF6j/k/KIa/txiyCbtVM0EYeFnAAAAFxBnuhFFSwj/wAAG591NV7/LyJ2HTW3VGansGMvyhY+o2EY2rK1PNh9Juk5LG9+DLbQRwINdcZLnfnzstphFnf/vGVLVPrsITepLWHwE4OuufL7DDANagHoacwy8QAAADcBnwd0R/8AACvRBkPfK6WRHYhPfJ+aJxijC6jE3fl+sFbw+W/XSM9LSldyZPHT7ivSgQfRptWwAAAAOAGfCWpH/wAALEQZjj3neg1BKilMkOJV7YGbk+fifxpbv71k0GLpJ9gIRUD6IqjfMEr+dxrivJLxAAAAXEGbDkmoQWyZTAhH//3hAAAFGZJg6GBqRs1/Bgi9VJWq9swO13bzmEwakmZMpTQr4dgoE7qvjFIufyrc+otRglRL1NbBSrhVuRzYAIyIU0U78USIdvnmm2mSNXxBAAAAU0GfLEUVLCP/AAAbk8PWl2dzvvwyunoO4eQR9vRLGSwzJu+53EoW7WwtIALZrKw2crqXslDf6IwDJ6hKmBjQtOKLTyeqTdCQjlvMXs+oVtnO7/KDAAAARQGfS3RH/wAALD+eoKMg8ehh4OOcce8l7kcWvImWP3p9e+tMKioDCDIHgATqlPUc29r/CgWeTq18pWc9N02FNbNxHSwFqQAAADMBn01qR/8AACxXM5T+xTt1Xcze6XYaQB+lCgZBw7RjORvLnchGF9TqKrIS9NMlVjFhVS4AAABbQZtQSahBbJlMFEx//IQAABRk+yYQiLwtbDcmDzRscedGAALG0UrGrTJKB09uFys8oazgSjHYUlp9tzwcsF3eeDaomkOh7axjy5qc6ihQj+iCn6rl5hJJW0G1RAAAADABn29qR/8AAC15UrbLstB7dAGRc8CA7F9TRpB713MGP6XNjWcca4GdY3SmdiFBBuEAABO/bW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAHFwAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAEul0cmFrAAAAXHRraGQAAAADAAAAAAAAAAAAAAABAAAAAAAAHFwAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAlgAAAGQAAAAAAAkZWR0cwAAABxlbHN0AAAAAAAAAAEAABxcAAACAAABAAAAABJhbWRpYQAAACBtZGhkAAAAAAAAAAAAAAAAAAAyAAABawBVxAAAAAAALWhkbHIAAAAAAAAAAHZpZGUAAAAAAAAAAAAAAABWaWRlb0hhbmRsZXIAAAASDG1pbmYAAAAUdm1oZAAAAAEAAAAAAAAAAAAAACRkaW5mAAAAHGRyZWYAAAAAAAAAAQAAAAx1cmwgAAAAAQAAEcxzdGJsAAAAnHN0c2QAAAAAAAAAAQAAAIxhdmMxAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAlgBkABIAAAASAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGP//AAAANmF2Y0MBZAAf/+EAGWdkAB+s2UCYM+XhAAADAAEAAAMAZA8YMZYBAAZo6+PLIsD9+PgAAAAAGHN0dHMAAAAAAAAAAQAAAWsAAAEAAAAAGHN0c3MAAAAAAAAAAgAAAAEAAAD7AAALCGN0dHMAAAAAAAABXwAAAAEAAAIAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAACAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAIAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAACAAAAAAEAAAMAAAAAAQAAAQAAAAABAAACAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAMAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAABxzdHNjAAAAAAAAAAEAAAABAAABawAAAAEAAAXAc3RzegAAAAAAAAAAAAABawAABGAAAABdAAAAJwAAAD0AAAA0AAAAfgAAADkAAAAfAAAAOQAAAHkAAABRAAAAOQAAADkAAABOAAAAJQAAADMAAAAdAAAAegAAACIAAAAhAAAAFgAAAFcAAAAjAAAAFgAAADAAAABOAAAANgAAADgAAAAyAAAAMQAAACwAAAAUAAAAJwAAABcAAAASAAAAFAAAAHEAAAAmAAAAOQAAAGwAAAAsAAAAKwAAAB8AAACDAAAALAAAAGsAAAAoAAAALAAAACoAAABTAAAAIgAAABkAAAAZAAAAVAAAACQAAAAbAAAAKwAAACgAAAAlAAAAGQAAABkAAABdAAAAcgAAADkAAAAnAAAAJAAAAIoAAAA6AAAAKAAAAD8AAABiAAAAQQAAAD8AAAAuAAAAhQAAADgAAAAtAAAAOAAAAFsAAABIAAAALwAAACsAAABiAAAAOAAAADoAAAAsAAAAbwAAAEMAAAAyAAAAOwAAAGQAAABKAAAAOQAAADMAAABRAAAAWgAAAEcAAAAwAAAAdQAAAEAAAAAkAAAAIQAAAGAAAAAsAAAAMQAAAGMAAAAwAAAALQAAAEYAAABZAAAAJwAAADgAAAAlAAAATgAAADsAAAAuAAAANgAAAGgAAABEAAAAOQAAAEwAAABhAAAAKwAAAE0AAAA0AAAAJwAAACsAAAA1AAAAQAAAACYAAAAlAAAASgAAADgAAAAjAAAAJwAAAG4AAABDAAAAIwAAADEAAABuAAAARAAAACkAAAA3AAAASgAAAD0AAAApAAAALQAAAHgAAABDAAAALgAAACwAAAB9AAAAOwAAAD8AAAA9AAAAjAAAAEgAAABOAAAAQwAAAGoAAABdAAAAOgAAAEEAAABxAAAAVQAAAEMAAAAvAAAAtwAAAE8AAAA/AAAAQQAAAJ4AAABNAAAAPAAAADkAAAB7AAAAOwAAAC0AAAA1AAAAswAAAEUAAAA5AAAAQAAAAIwAAABLAAAATQAAADgAAABwAAAAPQAAAEsAAABCAAAAbQAAAEwAAABVAAAATwAAAJcAAABFAAAATQAAAGkAAABFAAAASgAAAHsAAABGAAAAOgAAADsAAABjAAAASQAAAEUAAABKAAAAmwAAAFEAAAA5AAAAQgAAAHUAAABIAAAANgAAADkAAAB4AAAAOwAAADwAAABfAAAAQwAAACYAAAAsAAAAXgAAACkAAAA8AAAAmQAAAEMAAAAxAAAARAAAAIUAAABiAAAARAAAAE8AAAA9AAAASQAAAD4AAAB2AAAANAAAAJAAAABVAAAAOQAAADkAAABdAAAAQgAAAFAAAAByAAAAYwAAAE0AAAK3AAAAbwAAAE4AAABYAAAATwAAAJwAAABTAAAAnwAAAE8AAABfAAAAUQAAAIwAAABPAAAAUAAAAD8AAACwAAAASwAAAEQAAABgAAAAlwAAAEcAAABAAAAARAAAAJ4AAABQAAAANAAAADsAAABsAAAANQAAACcAAAAmAAAArQAAAEwAAAA7AAAASwAAAKoAAABvAAAAQQAAAD8AAABxAAAAXwAAADsAAAA2AAAAmwAAAFgAAABHAAAAOgAAAHgAAABcAAAAOgAAAEkAAACoAAAAVAAAAEkAAABFAAAAuAAAAGsAAABGAAAAUgAAAIgAAABSAAAASwAAAIkAAABKAAAAWQAAAEsAAACIAAAAXwAAAE0AAABNAAAAhwAAAGgAAABEAAAAUwAAAJMAAABCAAAASAAAAEoAAACNAAAATwAAAE8AAABNAAAAhgAAADAAAAB9AAAAiwAAADkAAAB3AAAArAAAAGUAAABMAAAAWwAAAJAAAABjAAAAjwAAAFwAAACZAAAAUAAAAEAAAACrAAAAbgAAAEYAAAA+AAAAmgAAAGAAAAA7AAAAPAAAAGAAAABXAAAASQAAADcAAABfAAAANAAAABRzdGNvAAAAAAAAAAEAAAAwAAAAYnVkdGEAAABabWV0YQAAAAAAAAAhaGRscgAAAAAAAAAAbWRpcmFwcGwAAAAAAAAAAAAAAAAtaWxzdAAAACWpdG9vAAAAHWRhdGEAAAABAAAAAExhdmY1OC41MS4xMDE=\" type=\"video/mp4\" />\n",
       "                 </video>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Animate learned policy\n",
    "save_dir='./videos/pytorch/reinforce/'\n",
    "env = make_env(env_name)\n",
    "generate_animation(env, save_dir=save_dir)\n",
    "[filepath] = glob.glob(os.path.join(save_dir, '*.mp4'))\n",
    "display_animation(filepath)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
